Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relative paths for -f are broken, and --project-directory appears to be stillborn #4710

Closed
Robin479 opened this issue Apr 6, 2017 · 17 comments

Comments

@Robin479
Copy link

Robin479 commented Apr 6, 2017

Our code-project is structure such that all the docker related stuff resides in a sub-folder ./docker including ./docker/docker-compose.yml and various files and folders (e.g. ./docker/nginx/nginx.conf) to be volume-mounted by docker-compose using relative paths.
Our build job used to invoke docker-compose from the project root (e.g. /workspace/project) with the -f ./docker/docker-compose.yml option, which started to fail since the update to 1.12.0, complaining that it couldn't find /workspace/project/nginx/ for mounting, which clearly should have been /workspace/project/docker/nginx/. I can change directory to /workspace/project/docker to avoid using the -f … option, to work around the problem. However, this clearly is a bug introduced in recent docker-compose, since nothing has changed in our project except the update to 1.12.0 (from 1.11.0, I guess).

ERROR: build path /workspace/project/nginx either does not exist, is not accessible, or is not a valid URL.

After some research I found, that 1.12.0 introduced the --project-directory option, so I tried docker-compose --project-directory ./docker from the project-root /workspace/project as before. It doesn't appear to work either, complaining that it couldn't find a suitable docker-compose.yml in this directory. Presumably, this directory mentioned is falsely assumed to be /workspace/project (=PWD), and docker-compose fails to properly check /workspace/project/docker as instructed (or maybe, I misunderstood the intended function of --project-directory …, in which case the -f … bug still applies).

ERROR:
Can't find a suitable configuration file in this directory or any
parent. Are you in the right directory?

    Supported filenames: docker-compose.yml, docker-compose.yaml

Using absolute paths as in docker-compose -f /workspace/project/docker/docker-compose.yml or docker-compose --project-directory /workspace/project/docker won't work either, except iff PWD=/workspace/project/docker, which renders -f and --project-directory completely useless.

I shall add that all mentioned paths are on the same file-system, and there are no soft-links involved.

For now, the only way to work around these problems is to cd /workspace/project/docker before every (parameterless) docker-compose invocation.

@dazhao-msft
Copy link

@shin-
Copy link

shin- commented Apr 6, 2017

Thank you for the report, we'll take a look ASAP.

@shin-
Copy link

shin- commented Apr 7, 2017

I tried a simple reproduction of this based on your report but I can't seem to be able to reproduce. Here's what I'm doing:

$ tree
.
└── docker
    ├── docker-compose.yml
    └── nginx
        └── nginx.conf

2 directories, 2 files

$ cat docker/docker-compose.yml 
version: '2.1'
services:
  foo:
    image: nginx
    volumes:
      - ./nginx/:/conf
    command: cat /conf/nginx.conf

$ docker-compose -f docker/docker-compose.yml up
Starting docker_foo_1
Attaching to docker_foo_1
foo_1  | foo
foo_1  | bar
docker_foo_1 exited with code 0
$ docker-compose version
docker-compose version 1.12.0, build b31ff33
docker-py version: 2.2.1
CPython version: 2.7.12
OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016

Did I miss something? Are you able to share your docker-compose.yml so I can gather more info?

Thanks.

@Robin479
Copy link
Author

Robin479 commented Apr 7, 2017

It appears the error message is related to the build: ./nginx setting:

$ pwd
/home/kk/workspace/docker-compose-fail
$ tree
.
└── docker
    ├── docker-compose.yml
    └── nginx
        └── Dockerfile

2 directories, 2 files
$ cat docker/docker-compose.yml 
version: "2.1"

services:
  nginx:
    image: shd/nginx
    build: ./nginx
$ cat docker/nginx/Dockerfile 
FROM nginx:alpine

MAINTAINER SHD System-Haus-Dresden GmbH
$ docker-compose -f docker/docker-compose.yml config
ERROR: build path /home/kk/workspace/docker-compose-fail/nginx either does not exist, is not accessible, or is not a valid URL.
$ docker-compose version
docker-compose version 1.12.0, build b31ff33
docker-py version: 2.2.1
CPython version: 2.7.12
OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016

@Robin479
Copy link
Author

Robin479 commented Apr 7, 2017

After removing the build: ./nginx setting and adding a volume mount, I have this:

version: "2.1"

services:
  nginx:
    image: nginx:alpine
    volumes:
      - ./nginx/:/conf

And the error message is gone (the error is not, though):

$ docker-compose -f docker/docker-compose.yml config
networks: {}
services:
  nginx:
    image: nginx:alpine
    volumes:
    - /home/kk/workspace/docker-compose-fail/nginx:/conf:rw
version: '2.1'
volumes: {}

Note the erroneous path translation on the volume mount, which should have been /home/kk/workspace/docker-compose-fail/docker/nginx:/conf:rw, but it is missing the docker/ sub-path.

@shin-
Copy link

shin- commented Apr 7, 2017

Thank you for taking a closer look. I have a patch for this in #4713 if you're able to check it out and verify it works for you as well.

@dazhao-msft
Copy link

@shin- Do you have private bits with the fix somewhere? I'd like to try it out as well.

@shin-
Copy link

shin- commented Apr 7, 2017

@dazhao-msft For Windows, you can download the Appveyor artifact here: https://ci.appveyor.com/project/docker/compose/build/master-3146/artifacts

@dazhao-msft
Copy link

I've verified that the issue reported at https://developercommunity.visualstudio.com/content/problem/40763/docker-compose-using-incorrect-path.html also gets fixed by the change. Thanks for the quick turnaround!

@dazhao-msft
Copy link

@shin-
Joffrey, do you know when the fix will be integrated in Docker CE for Windows? I checked the latest version 17.05.0-ce-rc1-win8 (11189), and the fix is not there yet.

This issue completely blocks the users from using Docker CE for Windows with Visual Studio. We hope to get this fix in ASAP.

Thanks!
-David

@GAAOPS
Copy link

GAAOPS commented Apr 20, 2017

Hi @shin,
Thanks for the fix, after i have downloaded this binary image: https://ci.appveyor.com/project/docker/compose/build/master-3146/artifacts
and copied it to "C:\Program Files\Docker\Docker\resources\bin" , it works again for me. I'm using Visual Studio 2017 with docker support and i'm able to debug it again.

Thanks,
Ghodrat

@friism
Copy link

friism commented Apr 21, 2017

@dazhao-msft this is targeted for 17.05!

@paulkling
Copy link

The fix from above fixed my problem as well.

@dlgoodchild
Copy link

I'm using Docker Toolkit for macOS, and I'm seeing this exact problem... but I'm on 17.06? and I'm up to date according to:

Docker 17.06.2-ce-mac27 is currently the newest version available.

$ docker-compose -v
docker-compose version 1.14.0, build c7bdf9e
$ docker -v
Docker version 17.06.2-ce, build cec0b72

What's the status of this issue with macOS?

Cheers

@shin-
Copy link

shin- commented Sep 22, 2017

@dlgoodchild The stable branch of Docker 4 Mac doesn't always include the latest version of Compose. While the issue reported here was fixed in 1.13, you might be running into another bug with similar symptoms. Please try with Compose 1.16.1 and open a new issue if you see the problem persist.

@nik131186
Copy link

nik131186 commented Dec 6, 2017

Not able to fix the below error. Any help in this matter is really appreciated.

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
The command '/bin/sh -c curl http://keplerproject.github.io/luarocks/releases/luarocks-2.2.2.tar.gz -O && tar -xzvf luarocks-2.2.2.tar.gz && cd luarocks-2.2.2 && ./configure && make build && make install' returned a non-zero code: 2

I tried to install unp on Mac so that it can support (tat.gz) file. But no luck so far

@Robin479
Copy link
Author

Robin479 commented Dec 7, 2017

@nik131186

  1. Your question is totally unrelated to this issue, i.e. off-topic!
  2. Your command should probably be quoted properly like this (please note where I put the single and double quotes): /bin/sh -c 'curl "http://keplerproject.github.io/luarocks/releases/luarocks-2.2.2.tar.gz" -O && tar -xzvf luarocks-2.2.2.tar.gz && cd luarocks-2.2.2 && ./configure && make build && make install'
  3. Your error message indicates, that the file you are trying to untar is actually not a tar.gz file despite its name. This can be verified by simply examining the file you just downloaded (e.g. with the file command): It is actually an HTML file with the server response of the curl request, which contains the message of a permanent redirect. In order for curl to follow the server redirect, you have to invoke it with -L.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants