sparc.client.services package

Submodules

sparc.client.services.o2sparc module

class sparc.client.services.o2sparc.O2SparcService(config: dict[str, Any] | SectionProxy | None = None, connect: bool = True)

Bases: ServiceBase

Wraps osparc python client library and fulfills ServiceBase interface

close() → None

Closes the osparc client.

connect() → ApiClient

Explicitily initializes client pool (not required)

get_profile() → str

Returns currently user profile.

Returns:

A string with username.

get_solver(solver_key: str, solver_version: str) → O2SparcSolver

Get a computational service (solver) to which jobs can be submitted.

Parameters:

solver_key :str

Solver key

solver_version :str

Solver version

Returns:

A O2SparcSolver object, to which jobs can be submitted

info() → str

Returns the version of osparc client.

set_profile(username: str, password: str) → str

Changes to a different user profile

Parameters:

username :str

API user key

password :str

API user secret

Returns:

A string with username.

class sparc.client.services.o2sparc.O2SparcSolver(api_client: ApiClient, solver_key: str, solver_version: str)

Bases: object

Wrapper for osparc.Solver

get_job_log(job_id: str) → TemporaryDirectory

Get the logs from a job

Parameters:

job_id: str

The job id

Returns:

A tempfile.TemporaryDirectory holding the log files

get_job_progress(job_id: str) → float

Get the job progress

Parameters:

job_id: str

The job id

Returns:

A float between 0.0 and 1.0 indicating the progress of the job. 1.0 means the job is done.

get_results(job_id: str) → dict[str, Any]

Get the results from a job

Parameters:

job_id: str

The job id

Returns:

A dictionary containing the results.

job_done(job_id: str) → bool

Job done

Parameters:

job_id: str

Job id

Returns:

A bool which is True if and only if the job is done

submit_job(job_inputs: dict[str, str | int | float | Path]) → str

Submit a job to the solver/computational service.

Parameters:

job_inputs: Dict[str, str | int | float | pathlib.Path]

When passing a file to the solver, pass it as a pathlib.Path object.

Returns:

A string representing the job id.

sparc.client.services.pennsieve module

class sparc.client.services.pennsieve.PennsieveService(config: dict | SectionProxy | None = None, connect: bool = False)

Bases: ServiceBase

A wrapper for the Pennsieve2 library

Parameters:

configdict

A configuration with defined profile name (pennsieve_profile_name).

connectbool

Determines if Sparc Client should initiate connection with Pennsieve Agent.

Attributes:

default_headersdict

A dictionary with headers to make HTTP requests.

host_apistr

A default HTTP address of the Pennsieve.

Pennsieveobject

A class holding st.

Methods:

connect()

Establishes connection with Pennsieve Agent.

info() -> str

Returns the version of Pennsieve Agent.

get_profile() -> str

Returns the currently used profile.

set_profile() -> str

Changes the profile to the specified name.

close()None

Closes Pennsieve Agent.

list_datasets(…)dict

Returns a dictionary with datasets matching search criteria.

list_files(…)dict

Returns a dictionary with datasets matching search criteria.

list_filenames(…)list

Returns a dictionary with filenames stored at AWS matching search criteria.

list_records(…)dict

Returns a dictionary with records matching search criteria.

Pennsieve: None = None

close() → None

Closes the Pennsieve Agent.

connect() → None

Establishes connection with Pennsieve Agent.

default_headers = {'Accept': 'application/json; charset=utf-8', 'Content-Type': 'application/json'}

delete(url: str, **kwargs)

Invokes DELETE endpoint on a server. Passing server name in url is optional.

Parameters:

urlstr

The address of the server endpoint to be called. The name of the server can be omitted.

kwargsdict

Additional information.

Returns:

String in JSON format with response from the server.

download_file(file_list: list[dict] | dict, output_name: str | None = None)

Downloads files into a local storage.

Parameters:

file_listlist[dict] or dict

Names of the file(s) to download with their parameters. The files need to come from a single database.

output_namestr

The name of the output file.

Returns:

A response from the server.

get(url: str, **kwargs)

Invokes GET endpoint on a server. Passing server name in url is optional.

Parameters:

urlstr

The address of the server endpoint to be called (e.g. api.pennsieve.io/datasets). The name of the server can be ommitted.

kwargsdict

A dictionary storing additional information.

Returns:

String in JSON format with response from the server.

Example:

p=Pennsieve() p.get(’https://api.pennsieve.io/discover/datasets’, params={‘limit’:20})

get_profile() → str

Returns currently used profile.

Returns:

A string with username.

host_api = 'https://api.pennsieve.io'

info() → str

Returns the version of Pennsieve Agent.

list_datasets(limit: int = 10, offset: int = 0, query: str | None = None, organization: str | None = None, organization_id: int | None = None, tags: List[str] | None = None, embargo: bool | None = None, order_by: str | None = None, order_direction: str | None = None) → list

Gets datasets matching specified criteria.

Parameters:

limitint

Max number of datasets returned.

offsetint

Offset used for pagination of results.

querystr

Fuzzy text search terms (refer to elasticsearch).

organizationstr

Publishing organization.

organization_idint

Publishing organization id.

tagslist(str)

Match dataset tags.

embargobool

Include embargoed datasets.

order_bystr

Field to order by:

name - dataset name date - date published size - size of dataset relevance - order determined by elasticsearch

order_directionstr

Sort order:

asc - Ascending, from A to Z desc - Descending, from Z to A

Returns:

A json with the results.

list_filenames(limit: int = 10, offset: int = 0, file_type: str | None = None, query: str | None = None, organization: str | None = None, organization_id: int | None = None, dataset_id: int | None = None) → list

Calls list_files() and extracts the names of the files. .. seealso:: list_files

list_files(limit: int = 10, offset: int = 0, file_type: str | None = None, query: str | None = None, organization: str | None = None, organization_id: int | None = None, dataset_id: int | None = None) → list

Parameters:

limitint

Max number of datasets returned.

offsetint

Offset used for pagination of results.

file_typestr

Type of file.

querystr

Fuzzy text search terms (refer to elasticsearch).

modelstr

Only return records of this model.

organizationstr

Publishing organization.

organization_idint

Publishing organization id.

dataset_idint

Files within this dataset.

Returns:

List of files stored at AWS with their parameters.

list_records(limit: int = 10, offset: int = 0, model: str | None = None, organization: str | None = None, dataset_id: int | None = None) → list

Parameters:

limitint

Max number of datasets returned.

offsetint

Offset used for pagination of results.

modelstr

Only return records of this model.

organizationstr

Publishing organization.

dataset_idint

Files within this dataset.

post(url: str, json: dict, **kwargs)

Invokes POST endpoint on a server. Passing server name in url is optional.

Parameters:

urlstr

The address of the server endpoint to be called (e.g. api.pennsieve.io/datasets). The name of the server can be omitted.

jsondict

A request payload with parameters defined by a given endpoint.

kwargsdict

Additional information.

Returns:

String in JSON format with response from the server.

profile_name: str = None

put(url: str, json: dict, **kwargs)

Invokes PUT endpoint on a server. Passing server name in url is optional.

Parameters:

urlstr

The address of the server endpoint to be called (e.g. api.pennsieve.io/datasets). The name of the server can be omitted.

jsondict

A request payload with parameters defined by a given endpoint.

kwargsdict

Additional information.

Returns:

String in JSON format with response from the server.

set_profile(profile_name: str) → str

Changes the profile to the specified name.

Parameters:

profile_namestr

The name of the profile to change into.

Returns:

A string with confirmation of profile switch.

sparc.client.services.metadata module

class sparc.client.services.metadata.MetadataService(config: dict | SectionProxy | None = None, connect: bool = False)

Bases: ServiceBase

A wrapper for the Elasticsearch Metadata library

Parameters:

configdict

A configuration containing necessary API key (scicrunch_api_key).

connectbool

Not needed with REST metadata services.

Attributes:

default_headersdict

A dictionary with headers to make HTTP requests.

host_apistr

A default HTTP address of the SciCrunch Elasticsearch API endpoint.

Methods:

get_profile() -> str

Returns the currently used API Key.

set_profile() -> str

Changes the API Key.

close()None

Not needed with REST metadata services.

getURL(…)dict

Supporting function to retrieve data from REST endpoint via GET This support Elasticsearch URL based queries

postURL(…)dict

Supporting function to retrieve data from REST endpoint This supports Elasticsearch JSON queries

list_datasets(…)dict

Returns a dictionary with datasets metadata.

search_datasets(…)dict

Returns a dictionary with datasets matching search criteria.

close() → None

Not needed as metadata services are REST service calls

connect() → str

Not needed as metadata services are REST service calls

default_headers = {'Accept': 'application/json; charset=utf-8', 'Content-Type': 'application/json'}

getURL(url, headers='NONE')

get_profile() → str

Returns currently used API key.

Returns:

A string with API Key.

info() → str

Returns information about the metadata search services.

list_datasets(limit: int = 10, offset: int = 0) → list

Lists datasets and associated metadata.

Parameters:

limitint

Max number of datasets returned.

offsetint

Offset used for pagination of results.

Returns:

A json with the results.

postURL(url, body, headers='NONE')

profile_name: str = None

scicrunch_api_key: str = None

search_datasets(query: str = '{"query": { "match_all": {}}}') → list

Gets datasets matching specified query.

This function provides

Parameters:

querystr

Elasticsearch JSON query.

Returns:

A json with the results.

set_profile(api_key: str) → str

Changes the API key to the specified name.

Parameters:

api_keystr

The API key to use.

Returns:

A string with confirmation of API key switch.

Module contents