Usage¶
Package¶
To use Magpie in a project, first you need to install it. To do so, you can do a basic pip install
.
For more details or other installation variants and environment preparation, see Installation and
Configuration procedures.
After this, you should be able to import the Python package to validate it is installed properly using:
import magpie
Web Application¶
In most situation, you will want to run Magpie as a Web Application in combination with some Web Proxy (e.g.: Twitcher) that can interrogate Magpie about applicable user authentication and permission authorization from the HTTP request session. To start the application, you can simply run the following command.
make start
This will first install any missing dependencies in the current environment (see Installation), and will after
start a basic Web Application on localhost:2001
with default configurations. Please note that you MUST have a
PostgreSQL database connection configured prior to running Magpie for it to operate (refer to Configuration
for details).
For running the application, multiple WSGI HTTP Servers can be employed (e.g.: Gunicorn, Waitress, etc.). They usually all support as input an INI configuration file for specific settings. Magpie also employs such INI file to customize its behaviour. See Configuration for further details, and please refer to the employed WSGI application documentation of your liking for their respective setup requirements.
API¶
When the application is started, the OpenAPI should be available under /api
path. This will render the
current API version and applicable requests. Please refer to this documentation to discover all provided
API paths and operations supported by Magpie on a running instance (that could be older than latest code
base). Alternatively, documentation of all versions is available on ReadTheDocs at the OpenAPI Specification location.
To select another version than the latest, first employ the ReadTheDocs version selector, then navigate to the
Swagger UI page, or directly adjust the URL as /en/{version}/api.html
.
The API allows an administrator-level user to modify services, resources, users and groups references via HTTP requests. To do these kind of operations, sufficient access rights must be provided to the corresponding user (either directly or through administrative-level group membership).
Some API routes are accessible by any-level user access. These are designated by Logged User in the documentation. When accessing such API paths, the applicable user for which the request is accomplished uses contextual information from authentication headers and/or cookies of the request. When no user is authenticated, a minimal subset of paths will provide some publicly available details, such as current session user. Other routes will be more verbose according to the applicable user permission (or is inherited group memberships).
Changed in version 2.0: As of this version, some API paths will offer additional Logged User operations such as self-registration to publicly available groups. See the appropriate API version documentation for routes that could be added or adjusted with this new functionality. Note also that a valid user account will still be required to access these routes.
Please refer to Permissions for further details about applicable user access levels.
CLI¶
After successful Installation of Magpie package, multiple Magpie CLI Helpers become available as CLI applications callable from the shell. These can be quite useful to run typical Magpie operations targeting a local or remote instance. Please refer to the relevant page for further details.
GUI¶
When the application is started, Magpie’s UI should be directly accessible on the top endpoint path. This interface allows quicker editing of elements accessible through the API by providing common operations such as modifying service fields or adjusting specific user-resource permissions. To have access to this interface, the user must have administrator permissions.
Changed in version 2.0: User-scoped views such as logged-user account details are now accessible to non-administrator level users.
These offer some basic functionalities such as registration to publicly visible groups. Users minimally require
to be logged-in (successful Authentication) in order to access these pages. The UI pages are accessible
using the Account
button from the main entrypoint of the Magpie UI.
Docker Application¶
Two Docker images are provided for very version released.
The first, simply named pavics/magpie
, is to execute Magpie itself as a Web Application.
This simultaneously offers both API and GUI interfaces accessible from the configured endpoint.
Using the same image and an override of the CMD
, it is also possible to run any of the CLI operations
with all preinstalled package dependencies. This image’s Dockerfile is a good reference to understand how to run the
Web Application locally if desired.
The second Docker image consists of Twitcher code base with integrated magpie.adapter.MagpieAdapter
such
that they can communicate between each other. Each tagged Magpie version will have an automatically deployed and
corresponding Twitcher image tag as pavics/twitcher:magpie-<version>
.
Usually, both images are employed in tandem with a PostgreSQL database connexion within a docker-compose.yml
configuration, similar to the following example. It is recommended to keep the versions in sync to ensure their
interoperability. Both images must also share some configurations, such as but not limited to, the same
MAGPIE_SECRET
in order to resolve operations in the same manner. See Twitcher Settings for
further details.
magpie:
image: pavics/magpie:<version>
# ... other Magpie configs ...
links:
- postgres
volumes:
- <some-path>/magpie.ini:/opt/local/src/magpie/config/magpie.ini
twitcher:
image: pavics/twitcher:magpie-<version>
# ... other Twitcher configs ...
links:
- postgres
volumes:
- <some-path>/twitcher.ini:/opt/birdhouse/src/twitcher/twitcher.ini
postgres:
image: postgres:9.6
# ... other PostgreSQL configs ...
environment:
PGDATA: /var/lib/postgresql/data/pgdata
volumes:
- <data-persistence-dir>:/var/lib/postgresql/data/pgdata
Note
Exposing the web application endpoints and ports to the outside world is left to the discretion of the developer
maintaining the server. This can be accomplished with basic mapping of ports using expose
keyword, but can
also be more securely achieved with links
and specific Nginx configurations to route locations as desired.
Testing¶
Basic Setup¶
To execute the tests on your local machine, a PostgreSQL instance must be running.
This PostgreSQL instance can be easily initialized by using the docker-compose.yml.example file
with the default values. Make sure the values defined in postgres.env
(custom copy of postgres.env.example)
are also corresponding to your testing setup.
There are both local and remote tests. The PostgreSQL is sufficient for the local tests but the remote tests also require a separately running instance of Magpie. Both suites are almost identical, but local ones are slightly more in depth as they can evaluate complicated edge cases hard to validate with a remote instance. The local tests should be your reference to see if new features are working and not breaking other functionalities. The remote are intended to validate older or pre-deployed servers, although with slightly more limited coverage.
A basic Magpie instance can also be initialized by using the docker-compose.yml.example file.
Note that the environment variables MAGPIE_TEST_REMOTE_SERVER_URL
and HOST_FQDN
must first be defined for the remote tests.
For example, these default values should work with docker-compose
:
export MAGPIE_TEST_REMOTE_SERVER_URL=http://localhost:2001
export HOST_FQDN=localhost
To start the tests, you can simply run one of the following command:
make test
make test-local
make test-remote
Note
Although targeted URL is localhost
, using MAGPIE_TEST_REMOTE_SERVER_URL
makes the test remote.
A Magpie instance should be running with that served location (e.g.: using make start
or docker-compose
).
Running local tests will boot its own Web Applicable instance, but PostgreSQL must still be accessible.
Customizing Tests¶
Specific tests can be enabled or disabled by category, using variables in magpie.env
(copy of magpie.env.example). Tests marked with the corresponding variables will be executed or not if they were
marked as matching that category. Alternatively, you can also provide the conditions directly using:
make test-custom SPEC="utils and local"
make test-custom SPEC="not remote"
# etc.
Value of SPEC
follows standard pytest
marker notations. Markers are mapped against the corresponding
MAGPIE_TEST_<MARKER>
variables in magpie.env.example.
When developing new features that change an older behaviour, tests MUST be made conditional to older versions to
preserve functional remote tests. To do so, the principal method is to employ utility tests.utils.warn_version
to
create a safeguard against execution of the test it is placed into. Tests can then be skipped if the targeted instance
does not fulfill version requirements specified with the safeguard.
Another approach is to employ if/else
blocks to separate execution by version of the tested instance as follows:
if TestVersion(self.version) >= TestVersion("1.2.3"):
# specific test for 1.2.3 and above
else:
# tests for an older instance
This allows fine-grained operation if the changes between versions are relatively “minor”.
Note
When developing new features, it often happens that the new version does not yet exist, since the feature is not
yet merged, tagged and deployed. Any safeguards checking against this future version would then always ignore
the new feature tests! To work around this, you should define variable MAGPIE_TEST_VERSION=latest
in your
environment (or any other loaded .env
file). This will tell the local test runner to consider the current
version as always greater that any compared minimal requirement, effectively enabling latest
test changes.