This vignette serves to document which sections of the openBIS API are implemented by the presented client package and which functionality was omitted. Furthermore, the basic mechanisms for making requests to openBIS are explained such that a user who wishes to add some of the omitted API functions has some information on how to extend infx. Some general documentation on the API, offered by the openBIS developers is available here.

Creating a REST call

The basic functionality powering all Representational state transfer (REST) calls of infx is provided by do_requests_serial() and do_requests_parallel(). Both functions take roughly the same input and produce identical results, but differ in how curl is called. The former function makes requests in a sequential fashion, iteratively calling curl::curl_fetch_memory() and the latter performs asynchronous requests, using curl::curl_fetch_multi() to assemble all requests and curl::multi_run() to carry them out in asynchronous parallel fashion.

Much of the behavior of do_requests_serial() and do_requests_parallel() can be customized using three functions passed as arguments create_handle, check and finally. Additionally, do_requests_*() takes a vector of urls and an optional object bodies which is expected to be of the same length as urls. Urls can either be passed as a character vector or a list of unevaluated function calls which will be evaluated using base::eval() shortly before being used. Both functions support retrying failed requests up to n_try times.

For constructing the requests, both urls and bodies are iterated together. First, the function passed as create_handle, which receives as input the current entry of the bodies object is used to construct a curl handle using curl::create_handle(). Together with the current URL entry, curl::curl_fetch_*() is called and the resulting object is passed as first argument to the check function, which receives as second argument the current entry of the bodies object. The function passed as check argument should make sure the request completed successfully and in case of failure return a simpleError object, created by base::simpleError(). In case of success, it should return the part of the curl response object that is of further interest (most likely the content entry). Next the finally function is called on the object returned by check to do some final processing (e.g. parsing JSON or reading a binary file). If the check function signaled a failure and the number of available tries as specified by n_try is not used up, the request is made again. If the allowed number of tries is exceeded, a warning is issued.

Asynchronous requests are implemented by first adding n_con curl handles to a new multi handle and starting the downloads by calling curl::multi_run(). For each successful request, a new handle is added to the pool using the done callback function of curl::multi_add(). In conjunction with passing urls as unevaluated function calls, this helps with urls that have a limited lifetime in that the URL is only created right before being consumed. Instead of adding all requests at the same time and letting curl handle queuing, only n_con requests are handled by curl at any given time.

A very basic use of do_requests_serial() is shown in the following example. The default function of the create_handle argument creates a new clean curl handle with no options set. The default function of the check argument makes sure that the status code is equal to 200 and returns the content entry of the list returned by curl::curl_fetch_memory(). The default function for finally is base::identity(). In order to receive a human-readable result instead of a raw vector, a function process_raw() is created, which converts the raw vector to a character vector yielding a JSON string.

Note that a list of length 1 is returned by do_requests_serial(). This is because the do_requests_*() functions are vectorized over requests and therefore return a list with as many entries as requests. A slightly more involved example for creating a request using do_requests_serial() is as follows.

In order to customize the curl handle, a create_handle function is supplied, which receives for each request the corresponding entry of the object passed as bodies argument. The finally function in this example parses the returned JSON object into a list. Since for POST requests, httpbin mirrors the POST data, the initial json list is returned.

Creating a JSON-RPC request

Building on functionality offered by the do_requests_*() functions, make_requests() helps with constructing JSON-RPC calls. As per the JSON-RPC 2.0 specification, a request object is a JSON string with the following members1:

  • jsonrpc: A string specifying the version of the JSON-RPC protocol.
  • method: A string containing the name of the method to be invoked.
  • params: A structured value that holds the parameter values to be used during the invocation of the method.
  • id: An identifier established by the client.

In order to create a list of such request objects, the arguments methods, params, ids and version, which map to method, params, id and jsonrpc, respectively, are combined and converted to JSON. OpenBIS API endpoint urls are constructed with the helper function api_url() which can receive arguments api_endpoint and host_url, forwarded by make_requests(). The argument n_con is used to specify the maximal number of simultaneous connections made to the server.

As make_requests() is designed to construct several requests at the time, the arguments methods and params are vectorized. If any of the passed objects is of length 1, it is replicated using base::rep() to the required length such that all three objects are of the same length. Care has to be taken with the list passed as params such that its length corresponds to the number of requests. This means that if only a single set of parameters is passed, this list has to be wrapped by another list.

As a last argument, a finally function can be passed, which defaults to process_json(). This function first converts all typed JSON objects to json_class objects using as_json_class(), resolves object references using resolve_references() in order to make all json_class objects self-contained and then creates a json_vec object using as_json_vec(), therefore allowing S3 method dispatch on lists of json_class objects. For more information on the specifics of this, please refer to the vignette “JSON object handling”.

For single requests, a wrapper around make_requests() is available as make_request(). This function wraps the params argument in a list such that a list of length 1 is passed to make_requests() and returns the first entry of the list resulting from calling make_requests(). The following example shows how make_request() is used to implement the API method listProjects, which takes an access token passed as a list as its only argument.

To illustrate the used of make_requests(), the API method getDownloadUrlForFileForDataSet is implemented. This function generates a download URL for a file in a data set and requires a separate API call for each combination of data set and file path. In order to carry out several of these requests asynchronously, a list has to be passed as params argument such that each entry contains a list holding a login token, a dataset code and a file path.

In addition to the JSON-RPC calls enabled by make_request()/make_requests(), the do_requests_*() functions are also used for (asynchronous) file downloads in fetch_files(). Slightly different functions are used as create_handle and check arguments but apart from that the logic remains largely the same.

Any arguments passed as ... to make_requests() will be forwarded to api_url() which creates an API endpoint URL and passes this on to the do_requests_*() functions. As all functions that issue API calls use make_requests(), this makes it possible to not only target the InfectX openBIS instance, but arbitrary openBIS servers that support the v1 JSON-RPC API. The following example serves to illustrate how this mechanism can be used to access the openBIS demo.

api_url() can accept arguments api_endpoint, host_url and full_url and ignores any further arguments. The first argument offers a selection of hard-coded API endpoints and in combination with the second argument is used to access one of them on the specified openBIS host. In case a URL that is not supported by the api_endpoint selector is desired, the complete URL can be passed as full_url which will simply be returned by api_url().

Summary of API methods

Currently no methods from the API sections

are implemented as they mostly deal with modifying metadata and creating new aggregation reports. The main focus of this package is retrieving data and therefore only methods from the API sections

are currently available. A more detailed overview of what functionality is implemented in this API client is given in the following sections.

Interface IDssServiceRpcGeneric

Generic functionality for interacting with the DSS. More information available here.

Method name Status Description
createReportFromAggregationService skipped Create the report from the specified aggregation service.
createReportFromDataSets skipped Creates for the specified data sets a report.
deleteSessionWorkspaceFile skipped Delete a file or directory in the session workspace.
getDownloadUrlForFileForDataSet implemented Returns an URL from which the requested file.
getDownloadUrlForFileForDataSet implemented Returns an URL from which the requested file of the specified data set can be downloaded.
getDownloadUrlForFileForDataSetWithTimeout implemented Returns an URL from which the requested file.
getDownloadUrlForFileForDataSetWithTimeout implemented Returns an URL from which the requested file of the specified data set can be downloaded.
getFileFromSessionWorkspace skipped Download a file from the user’s session workspace.
getPathToDataSet skipped Get a path to the data set.
getValidationScript skipped Get the validation script for the specified data set type.
listAggregationServices skipped Returns metadata for all aggregation services.
listAllShares skipped Lists all shares.
listFilesForDataSet implemented Get an array of FileInfoDss objects that describe the file-system structure of the data set.
listFilesForDataSet implemented Get an array of FileInfoDss objects that describe the file-system structure of the data set.
listTableReportDescriptions skipped Returns meta data for all reporting plugins which deliver a table.
putDataSet skipped Upload a new data set to the DSS.
putFileSliceToSessionWorkspace skipped Upload a file slice to the user’s session workspace.
putFileToSessionWorkspace skipped Upload a new file to the user’s session workspace.
shuffleDataSet skipped Moves specified data set to specified share.

Interface IDssServiceRpcScreening

Public DSS API for screening. More information available here.

Method name Status Description
getFeatureList skipped Return the feature codes of a specified feature list for a specified feature vector data set
getImageTransformerFactoryOrNull skipped Returns the transformer factory for the specified channel and the experiment to which the specified data sets belong.
listAvailableFeatureCodes implemented For a given set of feature vector data sets provide the list of all available features.
listAvailableFeatures implemented For a given set of feature vector data sets provide the list of all available features.
listAvailableImageRepresentationFormats implemented Return image representation formats available for the specified image data sets.
listImageMetadata implemented For a given set of image data sets, provide all image channels that have been acquired and the available (natural) image size(s).
listImageReferences implemented Lists microscopy image references for specified data set and channels.
listImageReferences implemented Lists microscopy image references for specified data set and channel.
listPlateImageReferences implemented Lists plate image references for specified data set, list of well positions and channels.
listPlateImageReferences implemented Lists plate image references for specified data set, list of well positions and channel.
loadFeatures implemented Conceptually, for a given list of data well references (i.e.
loadFeaturesForDatasetWellReferences implemented Conceptually, for a given list of dataset well references (i.e.
loadImages skipped Provide images for specified data set, list of well positions, channel, and optional thumb nail size.
loadImages skipped Provide images for specified microscopy data set, channel and optional thumb nail size.
loadImages skipped Provide images for a given list of image references (given by data set code, well position, channel and tile).
loadImages skipped Provide images for a given list of image references (specified by data set code, well position, channel and tile).
loadImages skipped Provides images for the specified list of image references (specified by data set code, well position, channel and tile) and image selection criteria.
loadImages skipped Provides images for the specified list of image references (specified by data set code, well position, channel and tile) and specified image representation format.
loadImages skipped Provide images (PNG encoded) for a given list of image references (given by data set code, well position, channel and tile).
loadImages skipped Provide images for a given list of image references (specified by data set code, well position, channel and tile).
loadImagesBase64 implemented Returns the same images as loadImages(String, IDatasetIdentifier, List, String, ImageSize) but the result is a list of base64 encoded strings that contain the image data.
loadImagesBase64 implemented Returns the same images as loadImages(String, IDatasetIdentifier, String, ImageSize) but the result is a list of base64 encoded strings that contain the image data.
loadImagesBase64 implemented Returns the same images as loadImages(String, List) but the result is a list of base64 encoded strings that contain the image data.
loadImagesBase64 implemented Returns the same images as loadImages(String, List, boolean) but the result is a list of base64 encoded strings that contain the image data.
loadImagesBase64 implemented Returns the same images as loadImages(String, List, IImageRepresentationFormatSelectionCriterion…) but the result is a list of base64 encoded strings that contain the image data.
loadImagesBase64 implemented Returns the same images as loadImages(String, List, ImageRepresentationFormat) but the result is a list of base64 encoded strings that contain the image data.
loadImagesBase64 implemented Returns the same images as loadImages(String, List, ImageSize) but the result is a list of base64 encoded strings that contain the image data.
loadImagesBase64 implemented Returns the same images as loadImages(String, List, LoadImageConfiguration) but the result is a list of base64 encoded strings that contain the image data.
loadPhysicalThumbnails skipped The fast method to provide registered thumbnail images (without calculating them) for the specified list of image references (specified by data set code, well position, channel and tile) and specified image representation format.
loadPhysicalThumbnailsBase64 implemented Returns the same images as loadPhysicalThumbnails(String, List, ImageRepresentationFormat) but the result is a list of base64 encoded strings that contain the image data.
loadThumbnailImages skipped Provide thumbnail images for specified microscopy data set.
loadThumbnailImages skipped Provide thumbnail images for a given list of image references (specified by data set code, well position, channel and tile).
loadThumbnailImagesBase64 implemented Returns the same images as loadThumbnailImages(String, IDatasetIdentifier, List) but the result is a list of base64 encoded strings that contain the image data.
loadThumbnailImagesBase64 implemented Returns the same images as loadThumbnailImages(String, List) but the result is a list of base64 encoded strings that contain the image data.
saveImageTransformerFactory skipped Saves the specified transformer factory for the specified channel of the specified data.

Interface IGeneralInformationService

Service for retrieving general informations. More information available here.

Method name Status Description
filterDataSetsVisibleToUser skipped Returns a filtered list of allDataSets containing those data sets which are visible to userId.
filterExperimentsVisibleToUser skipped Returns a filtered list of allExperiments containing those experiments which are visible to userId.
filterSamplesVisibleToUser skipped Returns a filtered list of allSamples containing those samples which are visible to userId.
getDataSetMetaData implemented Returns meta data for all specified data sets.
getDataSetMetaData implemented Returns meta data for all specified data sets.
getDataStoreBaseURLs implemented Returns the download URL for the data store of specified data sets.
getDefaultPutDataStoreBaseURL implemented Returns the URL for the default data store server for this openBIS AS.
getMaterialByCodes implemented Returns the materials with specified identifiers (i.e.
getMetaproject skipped Returns all entities tagged with given metaproject.
getMetaprojectOnBehalfOfUser skipped Returns all entities tagged with given metaproject for specified user.
isSessionActive implemented Returns true if session with the specified token is still active, false otherwise.
listAttachmentsForExperiment skipped Lists attachments of specified experiment.
listAttachmentsForProject skipped Lists attachments of specified project.
listAttachmentsForSample skipped Lists attachments of specified sample.
listDataSets implemented Return all data sets attached to the given samples.
listDataSets implemented Return all data sets attached to the given samples with connections.
listDataSetsForExperiments implemented Return all data sets attached to the given experiments with connections.
listDataSetsForExperimentsOnBehalfOfUser skipped Return all data sets attached to the given experiments with connections that the user userId is allowed to see.
listDataSetsForSample implemented Return the data sets attached to the specified sample, optionally including child samples.
listDataSetsOnBehalfOfUser skipped Return all data sets attached to the given samples with connections that the user userId is allowed to see.
listDataSetTypes implemented Returns all data set types.
listDataStores implemented Lists all DSS server registered this openBIS server instance.
listExperiments implemented Return all experiments of the given type that belong to the supplied projects.
listExperiments implemented Return all experiments matching a specified set of identifiers.
listExperimentsHavingDataSets implemented Return all experiments of the given type that belong to the supplied projects and have registered data sets.
listExperimentsHavingSamples implemented Return all experiments of the given type that belong to the supplied projects and have registered samles.
listExperimentTypes implemented Returns all experiment types.
listMetaprojects skipped Lists all metaprojects belonging to current user.
listMetaprojectsOnBehalfOfUser skipped Lists all metaprojects belonging to specified user.
listNamedRoleSets skipped Returns all named role sets.
listProjects implemented Returns all available projects.
listProjectsOnBehalfOfUser skipped Returns all available projects that a particular user is allowed to see.
listSamplesForExperiment implemented Return all samples that belong to the supplied experiment.
listSamplesForExperimentOnBehalfOfUser skipped Return all samples that belong to the supplied experiment that are visible to user userId.
listSampleTypes implemented Returns all sample types.
listSpacesWithProjectsAndRoleAssignments skipped Returns all spaces of specified database instance enriched with their projects and role assignments.
listVocabularies skipped Returns all available vocabularies together with the contained terms.
logout implemented Logout the session with the specified session token.
searchForDataSets implemented Return all data sets matching specified search criteria.
searchForDataSetsOnBehalfOfUser skipped Return all data sets matching specified search criteria and visible to user userId.
searchForExperiments implemented Returns all experiments matching specified search criteria.
searchForMaterials implemented Returns all material fulfilling specified search criteria.
searchForSamples implemented Return all samples that match the search criteria.
searchForSamples implemented Return all samples that match the search criteria.
searchForSamplesOnBehalfOfUser skipped Return all samples that match the search criteria and that a particular user is allowed to see.
tryGetDataStoreBaseURL implemented Returns the download URL for the data store of specified data set or null if such data set does not exist.
tryToAuthenticateForAllServices implemented Tries to authenticate specified user with specified password.

Interface IScreeningApiServer

This interface is a part of the official public screening API. More information available here.

Method name Status Description
getDatasetIdentifiers implemented Converts a given list of dataset codes to dataset identifiers.
getExperimentImageMetadata implemented Returns aggregated metadata for all images/plates within one experiment.
getPlateMetadataList implemented Fetches the contents of a given list of plates.
getPlateSample implemented For a given plateIdentifier, return the corresponding Sample.
getWellSample implemented For a given wellIdentifier, return the corresponding Sample including properties.
listAvailableFeatureCodes skipped Groups the specified objects by a data store code and calls IDssServiceRpcScreening listAvailableFeatureCodes(String, List) method for each group of objects on appropriate data store server.
listAvailableFeatures skipped Groups the specified objects by a data store code and calls IDssServiceRpcScreening listAvailableFeatures(String, List) method for each group of objects on appropriate data store server.
listAvailableImageRepresentationFormats skipped Groups the specified objects by a data store code and calls IDssServiceRpcScreening listAvailableImageRepresentationFormats(String, List) method for each group of objects on appropriate data store server.
listExperiments implemented Return the list of all visible experiments, along with their hierarchical context (space, project).
listExperiments implemented Return the list of all experiments visible to user userId, along with their hierarchical context (space, project).
listFeatureVectorDatasets implemented For a given set of plates (given by space / plate bar code), provide the list of all data sets containing feature vectors for each of these plates.
listImageDatasets skipped For a given set of plates provide the list of all data sets containing images for each of these plates.
listImageMetadata skipped Groups the specified objects by a data store code and calls IDssServiceRpcScreening listImageMetadata(String, List) method for each group of objects on appropriate data store server.
listPlateMaterialMapping implemented For a given list of plates, return the mapping of plate wells to materials contained in each well.
listPlates implemented Return the list of all visible plates assigned to any experiment, along with their hierarchical context (space, project, experiment).
listPlates implemented Return the list of all plates assigned to the given experiment.
listPlateWells implemented For the given experimentIdentifier, find all plate locations that are connected to the specified materialIdentifier.
listPlateWells implemented For the given materialIdentifier, find all plate locations that are connected to it.
listPlateWells implemented For the given plateIdentifier find all wells that are connected to it.
listRawImageDatasets implemented For a given set of plates provide the list of all data sets containing raw images for each of these plates.
listSegmentationImageDatasets implemented For a given set of plates provide the list of all data sets containing segmentation images for each of these plates.
loadFeatures skipped Groups the specified objects by a data store code and calls IDssServiceRpcScreening loadFeatures(String, List, List) method for each group of objects on appropriate data store server.
loadFeaturesForDatasetWellReferences skipped Groups the specified objects by a data store code and calls IDssServiceRpcScreening loadFeaturesForDatasetWellReferences(String, List, List) method for each group of objects on appropriate data store server.
loadImagesBase64 skipped Groups the specified objects by a data store code and calls IDssServiceRpcScreening loadImagesBase64(String, List) method for each group of objects on appropriate data store server.
loadImagesBase64 skipped Groups the specified objects by a data store code and calls IDssServiceRpcScreening loadImagesBase64(String, List, boolean) method for each group of objects on appropriate data store server.
loadImagesBase64 skipped Groups the specified objects by a data store code and calls IDssServiceRpcScreening loadImagesBase64(String, List, IImageRepresentationFormatSelectionCriterion…) method for each group of objects on appropriate data store server.
loadImagesBase64 skipped Groups the specified objects by a data store code and calls IDssServiceRpcScreening loadImagesBase64(String, List, ImageRepresentationFormat) method for each group of objects on appropriate data store server.
loadImagesBase64 skipped Groups the specified objects by a data store code and calls IDssServiceRpcScreening loadImagesBase64(String, List, ImageSize) method for each group of objects on appropriate data store server.
loadImagesBase64 skipped Groups the specified objects by a data store code and calls IDssServiceRpcScreening loadImagesBase64(String, List, LoadImageConfiguration) method for each group of objects on appropriate data store server.
loadPhysicalThumbnailsBase64 skipped Groups the specified objects by a data store code and calls IDssServiceRpcScreening loadPhysicalThumbnailsBase64(String, List, ImageRepresentationFormat) method for each group of objects on appropriate data store server.
loadThumbnailImagesBase64 skipped Groups the specified objects by a data store code and calls IDssServiceRpcScreening loadThumbnailImagesBase64(String, List) method for each group of objects on appropriate data store server.
logoutScreening skipped Logout the session with the specified session token.
tryLoginScreening skipped Authenticates the user with a given password.