SRA Data Locator v1 API Documentation
The SRA Data Locator provides information and access for data files listed in SRA but stored by a cloud provider.
Retrieve Data
Description
Provides the links for accessing files as well as information including; file name, file type, and modification date.
URL
https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/Traces/sdl/1/retrieve?<URL Parameters>
Method
GET or POST
URL Parameters
-
acc:
(REQUIRED) SRA, WGS or RefSeq Accession (one or more).
-
meta-only:
(Valid values : "yes","no") Returns only meta information without locations or signed URL. Equivalent to "ls -lR" in Unix. Default value is "no".
-
location:
(REQUIRED if meta-only=no) client/data location to get data local to the user. Location should match "service" and "region" fields of locality output. Examples would be s3.us-east-1, gs.us etc.
The list is open ended and depends on locations where data submitted to NCBI ended up being stored.
-
ngc:
Required authorization file provided by NCBI for access to dbGAP protected data files. Not used for public data access or when parameter 'meta-only' set in 'yes'.
-
filetype:
Filter to remove all but the specified file types. Examples include "bam", "cram" , "bai", "crai", "vcf", "vcf_index" etc. Valid values are any term included
in the "type" fields of json output.
Sample Response
Successful Response
curl -s -X POST -F ngc="@prj_phs710EA_test.ngc" "https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/Traces/sdl/1/retrieve?acc=SRR2043623&acc=SRR2043622&acc=SRR867664&acc=SRR1219805&location=s3.us-east-1"
[
{
"accession": "SRR1219805",
"status": 200,
"message": "ok",
"files": [
{
"objectId": "10569885",
"objectType": "remote",
"name": "NA19031.mapped.ILLUMINA.bwa.LWK.low_coverage.20130415.bam",
"size": "29851641509",
"md5": "c800d4179eaef5e7c6970fefeed1670c",
"type": "bam",
"service": "s3",
"link": "https://1000genomes.s3.amazonaws.com/phase3/data/NA19031/alignment/NA19031.mapped.ILLUMINA.bwa.LWK.low_coverage.20130415.bam?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJBXEMGZZHIM6WKMQ%2F20180518%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180518T204807Z&X-Amz-Expires=216000&X-Amz-SignedHeaders=host&x-amz-request-payer=requester&X-Amz-Signature=51c9c391a79395968dd6470cc9df85167e81cdd2c62b3b2254e4acf64d59f017",
"region": "us-east-1",
"expirationDate": "2018-05-18T22:48:05Z"
}
]
},
{
"accession": "SRR2043622",
"status": 200,
"message": "ok",
"files": [
{
"objectId": "10576671",
"objectType": "remote",
"name": "NA12891.mapped.ILLUMINA.bwa.CEU.high_coverage_pcr_free.20130906.bam",
"size": "258100635109",
"md5": "99e01e546635c028f4ebec4a27ba0af2",
"type": "bam",
"service": "s3",
"link": "https://1000genomes.s3.amazonaws.com/phase3/data/NA12891/high_coverage_alignment/NA12891.mapped.ILLUMINA.bwa.CEU.high_coverage_pcr_free.20130906.bam?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJBXEMGZZHIM6WKMQ%2F20180518%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180518T204807Z&X-Amz-Expires=216000&X-Amz-SignedHeaders=host&x-amz-request-payer=requester&X-Amz-Signature=c432dcb336ab7d70a338ec80c7911b51581ad0a6986ba359eed69e818baf1676",
"region": "us-east-1",
"expirationDate": "2018-05-18T22:48:05Z"
}
]
},
{
"accession": "SRR2043623",
"status": 200,
"message": "ok",
"files": [
{
"objectId": "10576670",
"objectType": "remote",
"name": "NA12892.mapped.ILLUMINA.bwa.CEU.high_coverage_pcr_free.20130906.bam",
"size": "291876795499",
"md5": "ed1f04b9b80ceef5c2ce4b52b76eed2a",
"type": "bam",
"service": "s3",
"link": "https://1000genomes.s3.amazonaws.com/phase3/data/NA12892/high_coverage_alignment/NA12892.mapped.ILLUMINA.bwa.CEU.high_coverage_pcr_free.20130906.bam?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJBXEMGZZHIM6WKMQ%2F20180518%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180518T204807Z&X-Amz-Expires=216000&X-Amz-SignedHeaders=host&x-amz-request-payer=requester&X-Amz-Signature=a57652310d8774e39b7409d7a7456f87fa9367b163b9637b5135a764f0f5e9a9",
"region": "us-east-1",
"expirationDate": "2018-05-18T22:48:05Z"
}
]
},
{
"accession": "SRR867664",
"status": 404,
"message": "No data at given location.region"
}
]
Error Response
There are two types of errors. First one occurs if something went wrong: corrupted application, database error, incorrect or missed mandatory parameters.
In this case code 400 or 500 will be set in HTTP header and an explanation in the body:
curl -D - "https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/Traces/sdl/1/retrieve"
HTTP/1.1 400 Bad Request
...
Content-Type: application/json
{
"status": 400,
"message": "No accession to process"
}
Second one occurs when an accession cannot be resolved (the accession does not exist, no permission, no data at given locality):
curl "https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/Traces/sdl/1/retrieve?acc=abc&acc=SRR1219805"
[
{
"accession": "SRR1219805",
"status": 403,
"message": "Access denied - please request permission to access phs000710 / UR in dbGaP"
},
{
"accession": "abc",
"status": 404,
"message": "Cannot resolve accession"
}
]
Sample Searches
-
curl -s -X POST -F ngc="@prj_phs710EA_test.ngc" "https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/Traces/sdl/1/retrieve?acc=SRR2043623&acc=SRR2043622&acc=SRR867664&acc=SRR1219805&location=s3.us-east-1"
-
curl -s -X POST "https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/Traces/sdl/1/retrieve?acc=SRR2043623&acc=SRR2043622&acc=SRR867664&acc=SRR1219805&location=s3.us-east-1&meta-only=yes"
-
curl -s "https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/Traces/sdl/1/retrieve?acc=SRR000002&location=sra-ncbi.public&type=sra"
-
curl -s "https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/Traces/sdl/1/retrieve?acc=AAAG01000010.1&type=wgs&location=ncbi"
-
curl -s "https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/Traces/sdl/1/retrieve?acc=NA000000007.1&type=na&location=ncbi"
-
curl -s "https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/Traces/sdl/1/retrieve?acc=NC_000001.9&type=refseq&location=ncbi"
Notes
We are using utility "cURL" to run examples.
You can download credential file for testing.
Locality
Description
Provides the locality or area where the data is stored on the cloud environment as this is different depending on where the data submitted to NCBI ended up being stored.
URL
https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/Traces/sdl/1/locality?<URL Parameters>
Method
GET or POSTURL Parameters
-
acc:
Mandatory parameter, SRA, WGA, NA or RefSeq Accession (one or more).
-
filetype:
Pptional parameter, restriction on set of file type. examples include "bam", "cram" , "bai", "crai", "vcf", "vcf_index" etc. The list is open ended and it is the same term returned
by "type" fileds of retruned json output. So any term found in that part of output is suitable to use in filetype parameter.
Sample Response
Successful Response
curl -s -X POST "https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/Traces/sdl/1/locality?acc=SRR2043623&acc=SRR2043622&acc=SRR867664&acc=SRR1219805&location=s3.us-east-1"
[
{
"accession": "SRR1219805",
"status": 200,
"message": "ok",
"files": [
{
"objectId": "10569885",
"name": "NA19031.mapped.ILLUMINA.bwa.LWK.low_coverage.20130415.bam",
"size": "29851641509",
"md5": "c800d4179eaef5e7c6970fefeed1670c",
"type": "bam",
"modificationDate": "2014-04-07T18:17:02Z",
"locality": [
{
"service": "s3",
"region": "us-east-1"
},
{
"service": "gs",
"region": "us"
},
{
"service": "ftp-ncbi"
}
]
},
{
"objectId": "3954499",
"name": "SRR1219805.pileup",
"size": "383123157",
"md5": "675819b646d674d595190a410d4f31e5",
"type": "sra",
"modificationDate": "2016-01-16T21:55:49Z",
"locality": [
{
"service": "sra-ncbi",
"region": "dbgap"
}
]
}
]
},
{
"accession": "SRR2043622",
"status": 200,
"message": "ok",
"files": [
{
"objectId": "10576671",
"name": "NA12891.mapped.ILLUMINA.bwa.CEU.high_coverage_pcr_free.20130906.bam",
"size": "258100635109",
"md5": "99e01e546635c028f4ebec4a27ba0af2",
"type": "bam",
"modificationDate": "2013-09-05T08:33:16Z",
"locality": [
{
"service": "s3",
"region": "us-east-1"
},
{
"service": "gs",
"region": "us"
},
{
"service": "ftp-ncbi"
}
]
},
{
"objectId": "6667038",
"name": "SRR2043622.pileup",
"size": "477717977",
"md5": "361eb0e5f15c888975be9eb228d0af0b",
"type": "sra",
"modificationDate": "2016-12-06T08:59:44Z",
"locality": [
{
"service": "sra-ncbi",
"region": "dbgap"
}
]
}
]
},
{
"accession": "SRR2043623",
"status": 200,
"message": "ok",
"files": [
{
"objectId": "10576670",
"name": "NA12892.mapped.ILLUMINA.bwa.CEU.high_coverage_pcr_free.20130906.bam",
"size": "291876795499",
"md5": "ed1f04b9b80ceef5c2ce4b52b76eed2a",
"type": "bam",
"modificationDate": "2013-09-05T08:57:54Z",
"locality": [
{
"service": "s3",
"region": "us-east-1"
},
{
"service": "gs",
"region": "us"
},
{
"service": "ftp-ncbi"
}
]
},
{
"objectId": "4941154",
"name": "SRR2043623.pileup",
"size": "993762315",
"md5": "180391bf474ff1905331fba1b3a3cb4e",
"type": "sra",
"modificationDate": "2016-04-24T17:32:41Z",
"locality": [
{
"service": "sra-ncbi",
"region": "dbgap"
}
]
}
]
},
{
"accession": "SRR867664",
"status": 200,
"message": "ok",
"files": [
{
"objectId": "4643483",
"name": "SRR867664.pileup",
"size": "7224778",
"md5": "15e95f910031ac17d4e63eae17c72a2a",
"type": "sra",
"modificationDate": "2016-03-12T08:42:17Z",
"locality": [
{
"service": "sra-ncbi",
"region": "public"
}
]
}
]
}
]
Error Response
There are two types of errors. First one occurs if something went wrong: corrupted application, database error, incorrect or missed mandatory parameters.
In this case code 400 or 500 will be set in HTTP header and an explanation in the body:
curl -D - "https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/Traces/sdl/1/locality"
HTTP/1.1 400 Bad Request
...
Content-Type: application/json
{
"status": 400,
"message": "No accession to process"
}
Second one occurs when an accession cannot be resolved (the accession does not exist, no permission, no data at given locality):
[
{
"accession": "SRR1219805",
"status": 403,
"message": "Access denied - please request permission to access phs000710 / UR in dbGaP"
},
...
Sample Searches
curl -X POST -F ngc="@prj_phs710EA_test.ngc" "https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/Traces/sdl/1/locality?acc=SRR2043622&acc=SRR2043623"
Notes
We are using utility "cURL" to run examples.
You can download credential file for testing.