list_urls.RdIn order to download files from openBIS, download urls have to be generated
first, which can be done by calling list_download_urls(). This function
is used in fetch_files(), which iterates over the selected files, creating
download links and executing the downloads. All data store servers
registered to an openBIS instance are listed by list_datastores() and data
store server urls per data set can be queried by calling
list_datastore_urls().
list_download_urls(token, x, ...) # S3 method for character list_download_urls(token, x, path, timeout = NA, ...) # S3 method for DataSet list_download_urls(token, x, path, timeout = NA, ...) # S3 method for DatasetIdentifier list_download_urls(token, x, path, timeout = NA, ...) # S3 method for DatasetReference list_download_urls(token, x, path, timeout = NA, ...) # S3 method for FeatureVectorDatasetReference list_download_urls(token, x, path, timeout = NA, ...) # S3 method for FeatureVectorDatasetWellReference list_download_urls(token, x, path, timeout = NA, ...) # S3 method for ImageDatasetReference list_download_urls(token, x, path, timeout = NA, ...) # S3 method for MicroscopyImageReference list_download_urls(token, x, path, timeout = NA, ...) # S3 method for PlateImageReference list_download_urls(token, x, path, timeout = NA, ...) # S3 method for DataSetFileDTO list_download_urls(token, x, timeout = NA, ...) list_datastores(token, ...) list_datastore_urls(token, x = NULL, ...) # S3 method for NULL list_datastore_urls(token, x, ...) # S3 method for character list_datastore_urls(token, x, ...) # S3 method for DataSet list_datastore_urls(token, x, ...) # S3 method for DatasetIdentifier list_datastore_urls(token, x, ...) # S3 method for DatasetReference list_datastore_urls(token, x, ...) # S3 method for FeatureVectorDatasetReference list_datastore_urls(token, x, ...) # S3 method for FeatureVectorDatasetWellReference list_datastore_urls(token, x, ...) # S3 method for ImageDatasetReference list_datastore_urls(token, x, ...) # S3 method for MicroscopyImageReference list_datastore_urls(token, x, ...) # S3 method for PlateImageReference list_datastore_urls(token, x, ...)
| token | Login token as created by |
|---|---|
| x | Object representing a (set of) dataset(s), e.g. a vector of dataset
codes, or a set of |
| ... | Generic compatibility. Extra arguments will be passed to
|
| path | A character vector of file paths within datasets. |
| timeout | Time-span (in seconds) for which the file download link should be valid. |
Both list_download_urls() and list_datastore_urls() return
character vectors while list_datastores() returns either a json_class
(single object) or a json_vec (multiple objects), dependent on the
number of resulting objects, with sub-type DataStore.
To specify files for which links are requested by list_download_urls(),
both a data set code and a file path are required. Objects, apart from
character vectors of data set codes, that may be passed to identify the
data set therefore include DataSet, DatasetIdentifier,
DatasetReference, FeatureVectorDatasetReference,
FeatureVectorDatasetWellReference, ImageDatasetReference,
MicroscopyImageReference and PlateImageReference. Additionally, dispatch
of list_download_urls() is possible on DataSetFileDTO objects which
contain both information on data set and file path of a file. A timeout
argument may be specified, determining how long (in seconds) the generated
url is valid for. If no specific timeout value is passed the url is valid
for what the openBIS documentation calls "a short time".
list_datastore_urls() as list_download_urls() ultimately requires a
character vector of data set codes to make the API call and therefore
dispatch is possible on, in addition to character vector, DataSet,
DatasetIdentifier, DatasetReference, FeatureVectorDatasetReference,
FeatureVectorDatasetWellReference, ImageDatasetReference,
MicroscopyImageReference and PlateImageReference objects. Dispatch on
NULL requests the default data store server url. Datastore sever url
related functionality is uninteresting for the InfectX set-up, as only a
single data store server exists, the url of which can be retrieved by a call
to list_datastores().
Other resource listing/downloading functions: fetch_images,
list_features, list_files
#> █─DataStore #> ├─code = DSS-SCREENING #> ├─downloadUrl = https://infectx.biozentrum.unibas.ch/datastore_server #> └─hostUrl = https://infectx.biozentrum.unibas.ch# search for a cell profiler feature data set from plate KB2-03-1I search <- search_criteria( attribute_clause("type", "HCS_ANALYSIS_CELL_FEATURES_CC_MAT"), sub_criteria = search_sub_criteria( search_criteria(attribute_clause("code", "/INFECTX_PUBLISHED/KB2-03-1I")), type = "sample" ) ) ds <- search_openbis(tok, search) # list all files of this data set files <- list_files(tok, ds) # extract file paths file_paths <- get_field(files, "pathInDataSet") # select a file file_path <- file_paths[grepl("Count_Cells", file_paths)] # generate url list_download_urls(tok, ds, file_path)#> [[1]] #> [1] "https://infectx.biozentrum.unibas.ch/datastore_server/stream-content?streamID=191205212826847-B0C1CC383B6A2A544A9989CD8C57C5F9" #> attr(,"data_set") #> [1] "20120629071528402-603231" #> attr(,"path") #> [1] "original/ibrain2_dataset_id_52954/Image.Count_Cells.mat" #># generate url and download file dat <- read_mat_files(url(list_download_urls(tok, ds, file_path)[[1L]])) attributes(dat)#> $object #> [1] "Image" #> #> $feature #> [1] "Count_Cells" #> #> $class #> [1] "url" "connection" #> #> $conn_id #> <pointer: 0x700> #>#> int [1:3456] 330 299 278 266 215 150 348 184 147 405 ...# set timeout to 2 sec file_url <- list_download_urls(tok, ds, file_path, timeout = 2L) tmp <- read_mat_files(url(file_url[[1L]])) # let timeout expire file_url <- list_download_urls(tok, ds, file_path, timeout = 2L) Sys.sleep(4L) tmp <- read_mat_files(url(file_url[[1L]]))#> Warning: Unknown endian: ru. Will assume Bigendian.#> Warning: Unknown MAT version tag: 28276. Will assume version 5.#> Warning: a read error occurred: #> Error in mat5ReadTag(this): Unknown data type. Not in range [1, 19]: 30062