magpie.cli.sync_resources

Synchronize local and remote resources.

To implement a new service, see magpie.cli.sync_services.SyncServiceInterface.

See also

• magpie.cli.sync_services

Module Contents

Functions

merge_local_and_remote_resources(...)	Main function to sync resources with remote server.
_merge_resources(→ magpie.typedefs.RemoteResourceTree)	Merge resources_local and resources_remote, adding the following keys to the output:
_sort_resources(→ None)	Sorts a nested resource dictionary.
_ensure_sync_info_exists(→ None)	Make sure the RemoteResourcesSyncInfo entry exists in the database.
_get_remote_resources(→ magpie.typedefs.JSON)	Request remote resources, depending on service type.
_delete_records(service_id, session)	Delete all RemoteResource based on a Service.resource_id.
_create_main_resource(→ None)	Creates a main resource for a service, whether one currently exists or not.
_update_db(→ None)	Writes remote resources to database.
_format_resource_tree(→ magpie.typedefs.RemoteResourceTree)
_query_remote_resources_in_database(...)	Reads remote resources from the RemoteResources table. No external request is made.
get_last_sync(→ Optional[datetime.datetime])	Obtain the date-time of the last known sync event for a service.
fetch_all_services_by_type(→ None)	Get remote resources for all services of a certain type.
fetch_single_service(→ None)	Get remote resources for a single service.
fetch(→ None)	Main function to get all remote resources for each service and write to database.
setup_cron_logger(→ None)
make_parser(→ argparse.ArgumentParser)
main(→ Any)	Main entry point for cron service.

Attributes

LOGGER
CRON_SERVICE
OUT_OF_SYNC

magpie.cli.sync_resources.LOGGER

magpie.cli.sync_resources.CRON_SERVICE = False

magpie.cli.sync_resources.OUT_OF_SYNC

magpie.cli.sync_resources.merge_local_and_remote_resources(resources_local: magpie.typedefs.RemoteResourceTree, service_sync_type: magpie.typedefs.Str, service_id: int, session: sqlalchemy.orm.session.Session) → magpie.typedefs.RemoteResourceTree

Main function to sync resources with remote server.

magpie.cli.sync_resources._merge_resources(resources_local: magpie.typedefs.RemoteResourceTree, resources_remote: magpie.typedefs.RemoteResourceTree, max_depth: int | None = None) → magpie.typedefs.RemoteResourceTree

Merge resources_local and resources_remote, adding the following keys to the output:

• remote_id: id of the RemoteResource

• matches_remote: True or False depending if the resource is present on the remote server

Returns:

Dictionary tree of the merged resources.

magpie.cli.sync_resources._sort_resources(resources: magpie.typedefs.RemoteResourceTree) → None

Sorts a nested resource dictionary.

The dictionary is expected to be valid as per sync_services.is_valid_resource_schema().

Returns:

None. Inplace modification.

magpie.cli.sync_resources._ensure_sync_info_exists(service_resource_id: int, session: sqlalchemy.orm.session.Session) → None

Make sure the RemoteResourcesSyncInfo entry exists in the database.

magpie.cli.sync_resources._get_remote_resources(service: magpie.models.Service) → magpie.typedefs.JSON

Request remote resources, depending on service type.

Parameters:

service – service for which to fetch remove children resources

Returns:

nested content as {"node_name": {"children": JSON, "resource_type": "resource_type"}}

magpie.cli.sync_resources._delete_records(service_id, session)

Delete all RemoteResource based on a Service.resource_id.

magpie.cli.sync_resources._create_main_resource(service_id: int, session: sqlalchemy.orm.session.Session) → None

Creates a main resource for a service, whether one currently exists or not.

Each RemoteResourcesSyncInfo has a main RemoteResource of the same name as the service. This is similar to the Service and Resource relationship.

magpie.cli.sync_resources._update_db(remote_resources: magpie.typedefs.RemoteResourceTree, service_id: int, session: sqlalchemy.orm.session.Session) → None

Writes remote resources to database.

magpie.cli.sync_resources._format_resource_tree(children: magpie.typedefs.ServiceResourceNodeTree) → magpie.typedefs.RemoteResourceTree

magpie.cli.sync_resources._query_remote_resources_in_database(service_id: int, session: sqlalchemy.orm.session.Session) → magpie.typedefs.RemoteResourceTree

Reads remote resources from the RemoteResources table. No external request is made.

Returns:

Dictionary of the form defined in sync_services.is_valid_resource_schema().

magpie.cli.sync_resources.get_last_sync(service_id: int, session: sqlalchemy.orm.session.Session) → datetime.datetime | None

Obtain the date-time of the last known sync event for a service.

magpie.cli.sync_resources.fetch_all_services_by_type(service_type: magpie.typedefs.Str, session: sqlalchemy.orm.session.Session) → None

Get remote resources for all services of a certain type.

Parameters:

• service_type – Service type for which all corresponding services will be synchronized.

• session – Database connexion to apply synchronization changes.

magpie.cli.sync_resources.fetch_single_service(service: magpie.models.Service | int, session: sqlalchemy.orm.session.Session) → None

Get remote resources for a single service.

Parameters:

• service – Specific service for which to synchronize remote resources.

• session – Database connexion to apply synchronization changes.

magpie.cli.sync_resources.fetch(settings: magpie.typedefs.SettingsType | None = None) → None

Main function to get all remote resources for each service and write to database.

magpie.cli.sync_resources.setup_cron_logger(log_level: magpie.typedefs.Str | int = logging.INFO) → None

magpie.cli.sync_resources.make_parser() → argparse.ArgumentParser

magpie.cli.sync_resources.main(args: Sequence[magpie.typedefs.Str] | None = None, parser: argparse.ArgumentParser | None = None, namespace: argparse.Namespace | None = None) → Any

Main entry point for cron service.