Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Incorrect docker container uris #281

Closed
sunitj opened this issue May 26, 2023 · 6 comments
Closed

Incorrect docker container uris #281

sunitj opened this issue May 26, 2023 · 6 comments
Labels
bug Something isn't working

Comments

@sunitj
Copy link

sunitj commented May 26, 2023 •
edited
Loading

Description of the bug

I seem to be getting errors like this with v1.1.1 with the docker profile:

nextflow.exception.ProcessUnrecoverableException: Task failed to start - CannotPullContainerError: Error response from daemon: pull access denied for nf-core/ubuntu, repository does not exist or may require 'docker login': denied: requested access to the resource is denied;

I tested by downloading nf-core/ubuntu manually, but I get the same error:

$ docker pull nf-core/ubuntu:20.04
Using default tag: latest
Error response from daemon: pull access denied for nf-core/ubuntu, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

To ensure that this wasn't an actual issue with my user. I also tried downloading the hello-world image, and that worked.

$ docker pull hello-world
Using default tag: latest
latest: Pulling from library/hello-world
719385e32844: Pull complete
Digest: sha256:fc6cf906cbfa013e80938cdf0bb199fbdbb86d6e3e013783e5a766f50f5dbce0
Status: Downloaded newer image for hello-world:latest
docker.io/library/hello-world:latest

Further investigation, seem to also indicate that URIs should start with quay.io/biocontainers/ start with biocontainers instead.

Command used and terminal output

No response

Relevant files

Here is a summary of all the incorrect URIs I could find and what I think they should be.

file current docker URI correct docker URI
nf-core/gunzip/main.nf 'nf-core/ubuntu:20.04' 'ubuntu:20.04'
nf-core/untar/main.nf 'nf-core/ubuntu:20.04' 'ubuntu:20.04'
nf-core/ampcombi/main.nf 'biocontainers/ampcombi:0.1.7--pyhdfd78af_0' 'quay.io/biocontainers/ampcombi:0.1.7--pyhdfd78af_0'
nf-core/ampir/main.nf 'biocontainers/r-ampir:1.1.0' 'quay.io/biocontainers/r-ampir:1.1.0'
nf-core/bioawk/main.nf 'biocontainers/bioawk:1.0--h5bf99c6_6' 'quay.io/biocontainers/bioawk:1.0--h5bf99c6_6'
nf-core/fargene/main.nf 'biocontainers/fargene:0.1--py27h21c881e_4' 'quay.io/biocontainers/fargene:0.1--py27h21c881e_4'
nf-core/fastqc/main.nf 'biocontainers/fastqc:0.11.9--0' 'quay.io/biocontainers/fastqc:0.11.9--0'
nf-core/multiqc/main.nf 'biocontainers/multiqc:1.14--pyhdfd78af_0' 'quay.io/biocontainers/multiqc:1.14--pyhdfd78af_0'
nf-core/prodigal/main.nf 'biocontainers/mulled-v2-2e442ba7b07bfa102b9cf8fac6221263cd746ab8:57f05cfa73f769d6ed6d54144cb3aa2a6a6b17e0-0' 'quay.io/biocontainers/mulled-v2-2e442ba7b07bfa102b9cf8fac6221263cd746ab8:57f05cfa73f769d6ed6d54144cb3aa2a6a6b17e0-0'
nf-core/prokka/main.nf 'biocontainers/prokka:1.14.6--pl5321hdfd78af_4' 'quay.io/biocontainers/prokka:1.14.6--pl5321hdfd78af_4'
nf-core/pyrodigal/main.nf 'biocontainers/mulled-v2-2fe9a8ce513c91df34b43a6610df94c3a2eb3bd0:697b3838b186fac6a9ceec198b09d4032162a079-0' 'quay.io/biocontainers/mulled-v2-2fe9a8ce513c91df34b43a6610df94c3a2eb3bd0:697b3838b186fac6a9ceec198b09d4032162a079-0'

System information

Nextflow: 23.04.1
Hardware: EC2 instance
Executor: local and AWS Batch
Container engine: Docker
OS: Amazon Linux 2
nf-core/funcscan version: 1.1.1
@sunitj sunitj added the bug Something isn't working label May 26, 2023
@jfy133
Copy link
Member

jfy133 commented May 31, 2023

I think this might be related to #279

@jfy133
Copy link
Member

jfy133 commented May 31, 2023

And by related, I mean solved

@jfy133
Copy link
Member

jfy133 commented May 31, 2023

@sunitj could you see if running with -r dev solves the problem? (x-post from slack)

@sunitj
Copy link
Author

sunitj commented Jun 1, 2023

@jfy133 I'm not sure if this is related, but I'm getting this error:

pigz: skipping: SPAdes-MetaBAT2-group-H1167_CA004_2060.45.fa is a symbolic link

More context:

Jun-01 17:53:54.392 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=NFCORE_FUNCSCAN:FUNCSCAN:PYRODIGAL (SPAdes-MetaBAT2-group-H1167_CA004_2060.45); work-dir=/mnt/efs/scratch/sunit/funcscan/TEST/20230601_dev/work/d1/3773d8c81bf52d679490e68c0ed00d
  error [nextflow.exception.ProcessFailedException]: Process `NFCORE_FUNCSCAN:FUNCSCAN:PYRODIGAL (SPAdes-MetaBAT2-group-H1167_CA004_2060.45)` terminated with an error exit status (1)
Jun-01 17:53:54.408 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'NFCORE_FUNCSCAN:FUNCSCAN:PYRODIGAL (SPAdes-MetaBAT2-group-H1167_CA004_2060.45)'

Caused by:
  Process `NFCORE_FUNCSCAN:FUNCSCAN:PYRODIGAL (SPAdes-MetaBAT2-group-H1167_CA004_2060.45)` terminated with an error exit status (1)

Command executed:

  pigz -cdf SPAdes-MetaBAT2-group-H1167_CA004_2060.45.fa > pigz_fasta.fna
  
  pyrodigal \
      -p meta   -g 11 \
      -i pigz_fasta.fna \
      -o SPAdes-MetaBAT2-group-H1167_CA004_2060.45.gff \
      -d SPAdes-MetaBAT2-group-H1167_CA004_2060.45.fna \
      -a SPAdes-MetaBAT2-group-H1167_CA004_2060.45.faa \
      -s SPAdes-MetaBAT2-group-H1167_CA004_2060.45.score
  
  pigz -nm SPAdes-MetaBAT2-group-H1167_CA004_2060.45*
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_FUNCSCAN:FUNCSCAN:PYRODIGAL":
      pyrodigal: $(echo $(pyrodigal --version 2>&1 | sed 's/pyrodigal v//'))
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  pigz: skipping: SPAdes-MetaBAT2-group-H1167_CA004_2060.45.fa is a symbolic link

Work dir:
  /mnt/efs/scratch/sunit/funcscan/TEST/20230601_dev/work/d1/3773d8c81bf52d679490e68c0ed00d

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

Running the .command.run separately:

$ cd /mnt/efs/scratch/sunit/funcscan/TEST/20230601_dev/work/d1/3773d8c81bf52d679490e68c0ed00d
$ bash .command.run
pigz: skipping: SPAdes-MetaBAT2-group-H1167_CA004_2060.45.fa is a symbolic link
pigz: skipping: SPAdes-MetaBAT2-group-H1167_CA004_2060.45.faa.gz exists
pigz: skipping: SPAdes-MetaBAT2-group-H1167_CA004_2060.45.faa.gz ends with .gz
pigz: skipping: SPAdes-MetaBAT2-group-H1167_CA004_2060.45.fna.gz exists
pigz: skipping: SPAdes-MetaBAT2-group-H1167_CA004_2060.45.fna.gz ends with .gz
pigz: skipping: SPAdes-MetaBAT2-group-H1167_CA004_2060.45.gff.gz exists
pigz: skipping: SPAdes-MetaBAT2-group-H1167_CA004_2060.45.gff.gz ends with .gz
pigz: skipping: SPAdes-MetaBAT2-group-H1167_CA004_2060.45.score.gz exists
pigz: skipping: SPAdes-MetaBAT2-group-H1167_CA004_2060.45.score.gz ends with .gz

Command used:

nextflow run nf-core/funcscan \
    -r dev \
    --input test_samplesheet.csv \
    --outdir CA004 \
    -profile docker \
    --run_bgc_screening \
    --save_databases \
    --bgc_skip_hmmsearch

@sunitj
Copy link
Author

sunitj commented Jun 1, 2023 •
edited
Loading

For a little more context, I was able to run the same test_samplesheet.csv using the same command (except the -r dev) without any issues on v1.1.0.

Command:

nextflow run nf-core/funcscan \
    -r 1.1.0 \
    --input test_samplesheet.csv \
    --outdir CA004 \
    -profile docker \
    --run_bgc_screening \
    --save_databases \
    --bgc_skip_hmmsearch

Output:

executor >  local (92)
[1e/dc357f] process > NFCORE_FUNCSCAN:FUNCSCAN:INPUT_CHECK:SAMPLESHEET_CHECK (test_samplesheet.csv)               [100%] 1 of 1 ✔
[b4/9080ff] process > NFCORE_FUNCSCAN:FUNCSCAN:GUNZIP_FASTA_PREP (SPAdes-MetaBAT2-group-H621_WT_Male.63.fa.gz)    [100%] 10 of 10 ✔
[56/43f7c8] process > NFCORE_FUNCSCAN:FUNCSCAN:BIOAWK (SPAdes-MetaBAT2-group-H621_CA004_SHAM.45)                  [100%] 10 of 10 ✔
[7c/7e04ca] process > NFCORE_FUNCSCAN:FUNCSCAN:PYRODIGAL (SPAdes-MetaBAT2-group-H621_CA004_SHAM.45)               [100%] 10 of 10 ✔
[3a/d7f8cd] process > NFCORE_FUNCSCAN:FUNCSCAN:BGC:UNTAR_CSS (css.tar.gz)                                         [100%] 1 of 1 ✔
[3e/389697] process > NFCORE_FUNCSCAN:FUNCSCAN:BGC:UNTAR_DETECTION (detection.tar.gz)                             [100%] 1 of 1 ✔
[a5/dda0cf] process > NFCORE_FUNCSCAN:FUNCSCAN:BGC:UNTAR_MODULES (modules.tar.gz)                                 [100%] 1 of 1 ✔
[33/a624c4] process > NFCORE_FUNCSCAN:FUNCSCAN:BGC:ANTISMASH_ANTISMASHLITEDOWNLOADDATABASES                       [100%] 1 of 1 ✔
[88/98c20c] process > NFCORE_FUNCSCAN:FUNCSCAN:BGC:ANTISMASH_ANTISMASHLITE (SPAdes-MetaBAT2-group-H1651_WT_IC.54) [100%] 10 of 10 ✔
[de/2fd151] process > NFCORE_FUNCSCAN:FUNCSCAN:BGC:DEEPBGC_DOWNLOAD                                               [100%] 1 of 1 ✔
[02/56a33c] process > NFCORE_FUNCSCAN:FUNCSCAN:BGC:DEEPBGC_PIPELINE (SPAdes-MetaBAT2-group-H621_CA004_SHAM.45)    [100%] 10 of 10 ✔
[0b/2b45d0] process > NFCORE_FUNCSCAN:FUNCSCAN:BGC:GECCO_RUN (MEGAHIT-MetaBAT2-group-H1074_CA004_2058.25)         [100%] 10 of 10 ✔
[c2/4ad49a] process > NFCORE_FUNCSCAN:FUNCSCAN:BGC:COMBGC (SPAdes-MetaBAT2-group-H621_CA004_SHAM.45)              [100%] 24 of 24 ✔
[0f/826419] process > NFCORE_FUNCSCAN:FUNCSCAN:CUSTOM_DUMPSOFTWAREVERSIONS (1)                                    [100%] 1 of 1 ✔
[60/ffbdab] process > NFCORE_FUNCSCAN:FUNCSCAN:MULTIQC                                                            [100%] 1 of 1 ✔
-[nf-core/funcscan] Pipeline completed successfully-
Completed at: 01-Jun-2023 19:27:50
Duration    : 37m 16s
CPU hours   : 6.8
Succeeded   : 92

@jfy133
Copy link
Member

jfy133 commented Jun 29, 2023

Should be fixed in: #279

1.1.2 coming imminentnyl!

@jfy133 jfy133 closed this as completed Jun 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sunitj @jfy133