Configuration

At startup, Magpie application will load multiple configuration files to define various behaviours or setup operations. These are defined through the configuration settings presented in below sections.

All generic Magpie configuration settings can be defined through either the magpie.ini file or environment variables. Values defined in magpie.ini are expected to follow the magpie.[variable_name] format, and corresponding MAGPIE_[VARIABLE_NAME] format is used for environment variables. Both of these alternatives match the constants defined in constants.py and can be used interchangeably.

Changed in version 1.1: Order of resolution will prioritize setting configurations over environment variables in case of duplicates resulting into different values. Environment variables will not override already specified setting values.

Previous versions of Magpie would instead prioritize environment variables, but this behaviour was deemed as counter intuitive. This is attributed to the global scope nature of environment variables that often made it hard to understand why some custom INI file would not behave as intended since those variable would inconsistently take precedence whether or not they were defined. Using a common configuration file makes it easier to maintain and understand the applied settings, and is therefore preferable.

Configuration Files

File: magpie.ini

This is the base configuration file that defines most of Magpie’s lower level configuration. A basic example is provided in magpie.ini which should allow any user to run the application locally. Furthermore, this file is used by default in each tagged Docker image. If you want to provide different configuration, the file should be overridden in the Docker image using a volume mount parameter, or by specifying an alternative path through the environment variable MAGPIE_INI_FILE_PATH.

File: magpie.env

By default, Magpie will try to load a magpie.env file which can define further environment variable definitions used to setup the application (see MAGPIE_ENV_FILE setting further below). An example of expected format and common variables for this file is presented in magpie.env.example.

Warning

If magpie.env cannot be found (e.g.: using setting MAGPIE_ENV_FILE) but magpie.env.example is available after resolving any previously set MAGPIE_ENV_DIR variable, this example file will be used to make a copy saved as magpie.env and will be used as the base .env file to load its contained environment variables. This behaviour is intended to reduce initial configuration and preparation of Magpie for a new user.

When loading variables from the .env file, any conflicting environment variable will NOT be overridden. Therefore, only missing but required values will be added to the environment to ensure proper setup of Magpie.

File: postgres.env

This file behaves exactly in the same manner as for magpie.env above, but for specific variables definition employed to setup the PostgreSQL database connection (see MAGPIE_POSTGRES_ENV_FILE setting below). File postgres.env.example and auto-resolution of missing postgres.env is identical to magpie.env case.

File: providers.cfg

This configuration file allows automatically registering Service definitions in Magpie at startup. When the application starts, it will look for corresponding services and add them to the database as required. It will also look for mismatches between the Service name and URL with the corresponding entry in the database to update it to the desired URL. See MAGPIE_PROVIDERS_CONFIG_PATH setting below to setup alternate references to this type of configuration. Please refer to the comment header of sample file providers.cfg for specific format and parameter details.

Changed in version 3.1: Some services, such as ServiceTHREDDS for instance, can take additional parameters to customize some of their behaviour. Please refer to Services chapter for specific configuration supported.

File: permissions.cfg

This configuration file allows automatically registering or cleaning Permission definitions in Magpie at startup. Each specified update operation is applied for the corresponding User and/or Group onto the specific Service or Resource. This file is processed after providers.cfg in order to allow permissions to be applied on freshly registered services. Furthermore, sub-resources are automatically created if they can be iteratively resolved with provided parameters of the corresponding permission entry. Resources should be defined using tree-path in this case, as described by format in magpie.api.management.resources.resources_utils.get_resource_path() or in example permissions.cfg.

See MAGPIE_PERMISSIONS_CONFIG_PATH setting below to setup alternate references to this type of configuration. Please refer to the comment header of sample file permissions.cfg for specific format details as well as specific behaviour of each parameter according to encountered use cases.

Configuration File Formats

Changed in version 1.9.2.

Any file represented in the Configuration chapter using any of the extension .cfg, .json, .yaml or .yml will be accepted interchangeably if provided. Both parsing as JSON and YAML will be attempted for backward compatibility of each resolved file path.

It is not mandatory for the name of each file to also match the employed name in the documentation, provided the paths can be resolved to valid files, though there is special handling of default .example extensions with matching file names when no other alternative configurations can be found. Again, this is mostly for backward compatibility.

Combined Configuration File

Added in version 2.0.

Since contents of all different configurations files (providers.cfg, permissions.cfg) reside under distinct top-level objects, it is actually possible to use an unique file to define everything. For example, one could define a combined configuration as follows.

# inside 'config.yml'

providers:
  some-service:
    url: http://${HOSTNAME}:8000
    title: Some Service
    public: true
    c4i: false
    type: api
    hooks: []   # see 'Service Hooks' section below for more details

groups:
  - name: my-group
    description: My Custom Group
    discoverable: false

users:
  - username: my-user
    group: my-group  # will reference above group

permissions:
  - service: api
    resource: /resource/user-resource  # will create both resources respecting children relationship
    type: route      # not mandatory here since service type 'api' only allows this type, but useful for other cases
    permission: read
    user: my-user    # will reference above user
    action: create
  - service: api
    resource: /groups
    permission: read
    group: my-group  # will reference above group
    action: create

webhooks:   # see 'Webhook Configuration' below for more details
  - name: <webhook_name>
    action: <webhook_action>
    method: GET | HEAD | POST | PUT | PATCH | DELETE
    url: <location>
    payload:
      <param_raw>: "value"              # some literal value that will be added to the payload as is
      <param_subst>: "{<param_value>}"  # <param_value> will be substituted (must be available for that action)
      ...

For backward compatibility reasons, Magpie will first look for separate files to load each section individually. To enforce using a combined file as above instead of the separate files, either provide MAGPIE_CONFIG_PATH = <path>/config.yml, or ensure that both environment variable MAGPIE_PROVIDERS_CONFIG_PATH and MAGPIE_PERMISSIONS_CONFIG_PATH specifically refer to this same YAML file. For all of these variables, magpie.[variable_name] formatted settings are also supported through definitions within magpie.ini.

When loading configurations from a combined file, the order of resolution of each section is the same as when loading definitions from multiple files, meaning that providers are first registered, followed by individual permissions, with the dynamic creation of any missing user or group during this process. If an explicit user or group definition can be found under the relevant sections, additional parameters are employed for their creation. Otherwise defaults are assumed and only the specified user or group name are employed. Please refer to files providers.cfg and permissions.cfg for further details about specific formatting and behaviour of each available field.

Added in version 3.6: The webhook section allows to define external connectors to which Magpie should send requests following certain events. These are described in further details in Webhook Configuration section.

Added in version 3.12: Variable MAGPIE_WEBHOOKS_CONFIG_PATH was added and will act in a similar fashion as their providers and permissions counterparts, to load definitions from multiple configuration files.

Service Hooks

Added in version 3.25.

Warning

Requires Twitcher version 0.7.0 minimally to use this feature. Versions 0.6.x of Twitcher remain compatible but will not call the adapter hooks as the feature did not exist at that point.

Under each Service within providers.cfg or the Combined Configuration File, it is possible to provide a section named hooks that lists additional pre/post request/response processing operations to apply when matched against the given request filter conditions. These hooks are plugin-based Python scripts that can modify the proxied request and response when Magpie and Twitcher work together using the utilities_adapter. Each Service Hook must be configured using the following parameters.

Service Hook configuration

Field	Requirement	Description
type	required	Literal string { request | response } of the desired instance where to invoke the hook.
path	required	Service-specific request path or regular expression pattern to be matched for invoking the hook. Path starts after Twitcher proxy prefix path and Service name (i.e.: path as if there was no proxy).
method	optional	Literal string { HEAD | GET | POST | PUT | PATCH | DELETE | * } (default: * representing any method). HTTP method that must be matched for invoking the hook.
query	optional	Request query string or regular expression pattern to be matched for invoking the hook (default: .*). Matches anything if not specified. To match explicitly no-query condition, provide an empty string ("").
target	required	Location of the function that will handle hook processing when request matching conditions are met. Path should be absolute or relative to MAGPIE_ROOT and must be a valid Python file. Path should include the function name using format: some/path/script.py:func.

More specifically, when a Service or children Resource is accessed, triggering a proxied request through Twitcher, the authenticated and authorized request goes through hooks processing chain that can adjust certain request and response parameters (e.g.: add headers, filter the body, etc.), or even substitute the request definition entirely based on target implementations. Hooks are applied in the same order as they are defined in the configuration when they match the inbound request, propagating the request/response across each Service Hook call. Plugin scripts can therefore apply some advanced logic to improve the synergy between the protected services. They can also be employed to apply some Service specific operations such as filtering protected contents that Magpie and Twitcher cannot themselves process evidently. Service Hook are applied after authentication and authorization of the request, just before sending the request to the real protected Service (in case of request hook), and just after receiving its response (in case of response hook.

Permitted signatures of Service Hook functions are as presented below. The first argument (request or response respectively) is always required. Its modified definition must be returned as well. The other parameters (service, hook, context) are all optional. They represent the specific configurations that triggered the target call. Optional arguments can be specified in any order or combination, but MUST use the exact argument names indicated below.

Changed in version 3.26.0: Added the context parameter.

def request_hook(request: pyramid.request.Request) -> pyramid.request.Request: ...

def request_hook(request: pyramid.request.Request,
                 service: magpie.typedefs.ServiceConfigItem,
                 hook: magpie.typedefs.ServiceHookConfigItem,
                 context: magpie.adapter.HookContext,
                 ) -> pyramid.request.Request: ...

def response_hook(response: pyramid.response.Response) -> pyramid.response.Response: ...

def response_hook(response: pyramid.response.Response,
                  service: magpie.typedefs.ServiceConfigItem,
                  hook: magpie.typedefs.ServiceHookConfigItem,
                  context: magpie.adapter.HookContext,
                  ) -> pyramid.response.Response: ...

Service Hook function parameters

Parameter	Type	Description
request/response	pyramid.request.Request / pyramid.response.Response	Applicable instance to be modified by the hook according to type definition.
service	magpie.typedefs.ServiceConfigItem	Definition of the Service as specified in the original configuration file.
hook	magpie.typedefs.ServiceHookConfigItem	Specific hook entry (under the above service definition), that was matched to call the target function.
context	magpie.adapter.HookContext	Attribute holder with internal handlers and references that were used for processing the current request. This offers precomputed accessors to magpie.adapter.MagpieAdapter, the contextual Service involved in the current transaction (i.e.: specific magpie.services.ServiceInterface implementation), as well as the underlying database reference magpie.models.Resource that represents the accessed Service. Using those definitions, it is possible for a Service Hook to perform further Permission verifications for manipulating the request or response (e.g.: filtering content).

Warning

When using the context parameter, it is important to consider that any operation performed that could involve additional database queries or Permissions Resolution steps could slow down the observed response speed from the protected Service. Optimizing or caching the result of such operations could become critical if the request that triggers the corresponding Service Hook require high demands.

See also

File providers.cfg presents contextual information and location of the hooks schema under example provider definitions.

File tests/hooks/request_hooks.py presents some examples of hook target functions with common operations to update request and response parameters.

Settings and Constants

Environment variables can be used to define all following configurations (unless mentioned otherwise with [constant] keyword). Most values are parsed as plain strings, unless they refer to an activatable setting (e.g.: True or False), or when specified with more specific [<type>] notation.

Configuration variables will be used by Magpie on startup unless prior definition is found within magpie.ini. All variables (i.e.: non-constant parameters) can also be specified by their magpie.[variable_name] setting counterpart as described at the start of the Configuration section.

Loading Settings

These settings can be used to specify where to find other settings through custom configuration files.

MAGPIE_MODULE_DIR

[constant]

Path to the top level magpie module (i.e.: source code).

MAGPIE_ROOT

[constant]

Path to the containing directory of Magpie. This corresponds to the directory where the repository was cloned or where the package was installed.

MAGPIE_CONFIG_DIR

(Default: ${MAGPIE_ROOT}/config)

Configuration directory where to look for providers.cfg and permissions.cfg files.

If more than one file for any of those individual type of configuration needs to be loaded from a directory, the MAGPIE_PROVIDERS_CONFIG_PATH and MAGPIE_PERMISSIONS_CONFIG_PATH must be employed instead. Setting this variable will only look for files named exactly as above, unless the more explicit definitions of MAGPIE_<type>_CONFIG_PATH variables are also provided.

Warning

This setting is ignored if MAGPIE_CONFIG_PATH is specified.

MAGPIE_PROVIDERS_CONFIG_PATH

(Default: ${MAGPIE_CONFIG_DIR}/providers.cfg)

Path where to find a providers.cfg file. Can also be a directory path, where all contained configuration files will be parsed for providers section and will be loaded sequentially.

Please refer to providers.cfg for specific format details and parameters.

Note

If a directory path is specified, the order of loaded configuration files is alphabetical. Matching Service will be overridden by files loaded last.

Changed in version 1.7.4: Loading order of multiple files was NOT guaranteed prior to this version.

This could lead to some entries to be loaded in inconsistent order.

Warning

This setting is ignored if MAGPIE_CONFIG_PATH is specified.

MAGPIE_PROVIDERS_HOOKS_PATH

(Default: MAGPIE_ROOT)

Defines the root directory were to look for target references in Service Hooks when the provided path is relative.

Note

When using the Docker image, the default MAGPIE_ROOT corresponds to the source location. When using the installed package (unless -e was provided to pip for development installation), this MAGPIE_ROOT will be located in site-packages of the target Python environment.

MAGPIE_PERMISSIONS_CONFIG_PATH

(Default: ${MAGPIE_CONFIG_DIR}/permissions.cfg)

Path where to find permissions.cfg file. Can also be a directory path, where all contained configuration files will be parsed for permissions section and will be loaded sequentially.

Please refer to permissions.cfg for specific format details of the various parameters.

Note

If a directory path is specified, the order of loaded configuration files is alphabetical.

Changed in version 1.7.4: Loading order of multiple files was NOT guaranteed prior to this version.

With older versions, cross-file references to services or resources should be avoided to ensure that, for example, any parent resource dependency won’t be missing because it was specified in a second file loaded after the first. Corresponding references can be duplicated across files and these conflicts will be correctly handled according to configuration loading methodology. Later versions are safe to assume alphabetical loading order.

Warning

This setting is ignored if MAGPIE_CONFIG_PATH is specified.

MAGPIE_WEBHOOKS_CONFIG_PATH

(Default: None)

Added in version 3.12.

Path where to find a file or a directory of multiple configuration files where webhooks section(s) that provide definitions for Webhook Configuration can be loaded from.

Examples of such configuration section is presented in the example Combined Configuration File. When multiple files are available from a directory path, they are loaded by name alphabetically.

Warning

This setting is ignored if MAGPIE_CONFIG_PATH is specified.

MAGPIE_CONFIG_PATH

Path where to find a combined YAML configuration file which can include providers, permissions, users and groups sections to sequentially process registration or removal of items at Magpie startup.

See Combined Configuration File for further details and an example of its structure.

Changed in version 3.6: The configuration can also contain a webhooks section, as described in Webhook Configuration and presented in the sample Combined Configuration File.

Warning

When this setting is defined, all other combinations of MAGPIE_CONFIG_DIR, MAGPIE_PERMISSIONS_CONFIG_PATH, MAGPIE_PROVIDERS_CONFIG_PATH and MAGPIE_WEBHOOKS_CONFIG_PATH are effectively ignored in favour of definitions in this file. It is not possible to employ the single Combined Configuration File at the same time as multi-configuration file loading strategy from a directory.

MAGPIE_INI_FILE_PATH

Specifies where to find the initialization file to run Magpie application.

Note

This variable ignores the setting/env-var resolution order since settings cannot be defined without firstly loading the file referenced by its value.

See also

config_magpie_ini

MAGPIE_ENV_DIR

(Default: "${MAGPIE_ROOT}/env")

Directory path where to look for .env files. This variable can be useful to load specific test environment configurations or to specify a local path while the actual Magpie code is located in a Python site-packages directory (.env files are not installed to avoid hard-to-resolve settings loaded from an install location).

MAGPIE_ENV_FILE

(Default: "${MAGPIE_ENV_DIR}/magpie.env")

File path to magpie.env file with additional environment variables to configure the application.

See also

File: magpie.env

MAGPIE_POSTGRES_ENV_FILE

(Default: "${MAGPIE_ENV_DIR}/postgres.env")

File path to postgres.env file with additional environment variables to configure the postgres connection.

See also

File: postgres.env

Application Settings

Following configuration parameters are used to define values that are employed by Magpie after loading the Loading Settings. All magpie.[variable_name] counterpart definitions are also available as described at the start of the Configuration section.

MAGPIE_URL

(Default: "http://localhost:2001")

Full hostname URL to use so that Magpie can resolve his own running instance location.

Note

If the value is not set, Magpie will attempt to retrieve this critical information through other variables such as MAGPIE_HOST, MAGPIE_PORT, MAGPIE_SCHEME and HOSTNAME. Modifying any of these variables partially is permitted but will force Magpie to attempt building the full URL as best as possible from the individual parts. The result of these parts (potential using corresponding defaults) will have the following format: "${MAGPIE_SCHEME}//:${MAGPIE_HOST}:${MAGPIE_PORT}".

Note

The definition of MAGPIE_URL or any of its parts to reconstruct it must not be confused with parameters defined in the [server:main] section of the provided magpie.ini configuration. The purpose of variable MAGPIE_URL is to define where the exposed application is located, often representing the server endpoint for which the Magpie instance is employed. The values of host and port, or bind defined in [server:main] instead correspond to how the WSGI application is exposed (e.g.: through Gunicorn), and so represents a local web application that must be mapped one way or another to the server when running within the Docker Application.

MAGPIE_SCHEME

(Default: "http")

Protocol scheme URL part of Magpie application to rebuild the full MAGPIE_URL.

MAGPIE_HOST

(Default: "localhost")

Domain host URL part of Magpie application to rebuild the full MAGPIE_URL.

MAGPIE_PORT

[int] (Default: 2001)

Port URL part of Magpie application to rebuild the full MAGPIE_URL.

MAGPIE_CRON_LOG

(Default: "~/magpie-cron.log")

Path that the cron operation should use for logging.

MAGPIE_LOG_LEVEL

(Default: INFO)

Logging level of operations. Magpie will first use the complete logging configuration found in magpie.ini in order to define logging formatters and handler referencing to the logger_magpie section. If this configuration fail to retrieve an explicit logging level, this configuration variable is used instead to prepare a basic logger, after checking if a corresponding magpie.log_level setting was instead specified.

Warning

When setting DEBUG level or lower, Magpie will potentially dump some sensitive information in logs such as access tokens. It is important to avoid this setting for production systems.

MAGPIE_LOG_PRINT

[bool] (Default: False)

Specifies whether Magpie logging should also enforce printing the details to the console when using Magpie CLI Helpers. Otherwise, the configured logging methodology in magpie.ini is used (which can also define a console handler).

MAGPIE_LOG_REQUEST

[bool] (Default: True)

Specifies whether Magpie should log incoming request details.

Note

This can make Magpie quite verbose if large quantity of requests are accomplished.

MAGPIE_LOG_EXCEPTION

[bool] (Default: True)

Specifies whether Magpie should log a raised exception during a process execution.

MAGPIE_SMTP_USER

(Default: "Magpie")

Added in version 3.13.

Display name employed as sending user of notification emails.

If explicitly overridden by an empty string, the MAGPIE_SMTP_FROM is used as replacement.

MAGPIE_SMTP_FROM

(Default: None)

Added in version 3.13.

Email that identifies the sender of notification emails by the application.

This value is also employed to run the authentication step to the SMTP server in combination with MAGPIE_SMTP_PASSWORD if it is also provided. Furthermore, if the value is provided while MAGPIE_SMTP_USER is empty, the default email sender (display name) will revert to this value.

MAGPIE_SMTP_PASSWORD

(Default: None)

Added in version 3.13.

Authentication password to use in combination with MAGPIE_SMTP_FROM to connect the server specified by MAGPIE_SMTP_HOST as required.

Leave blank if SMTP server does not require or should not execute authentication step.

MAGPIE_SMTP_HOST

Added in version 3.13.

Host of the SMTP server to employ for sending notification emails.

MAGPIE_SMTP_PORT

[int] (Default: 465)

Added in version 3.13.

Port of the outgoing notification emails from the SMTP server.

In case of doubt, port value 25 (an sometimes 587) is employed for non-encrypted emails. For secure TLS, 587 is the usual choice, and 465 when using SSL. Other ports based on the functionalities offered by targeted MAGPIE_SMTP_HOST could be available.

Note that MAGPIE_SMTP_SSL should be set accordingly when using those standard values. It is strongly recommended to employ an encrypted email since transferred details by Magpie can potentially contain some sensible details.

MAGPIE_SMTP_SSL

[bool] (Default: True)

Added in version 3.13.

Specifies if SSL should be employed for sending email.

If not enabled, Magpie will first attempt to establish a TLS connection if the targeted SMTP server supports it to use encrypted emails. If it is not supported by that server, it falls back to unencrypted emails since no other alternatives exist.

MAGPIE_TOKEN_EXPIRE

[int] (Default: 86400 seconds)

Added in version 3.7.

Duration for which temporary URL tokens will remain valid until automatically removed.

These tokens can be used for many different applications within Magpie, but are notably employed for handling callback URL operations in tandem with a given Webhook (see also: Webhook Actions).

MAGPIE_UI_ENABLED

[bool] (Default: True)

Specifies whether Magpie graphical user interface should be available with the started instance. If disabled, all routes that normally refer to the UI will return 404, except the frontpage that will return a simple JSON description as it is normally the default entrypoint of the application.

MAGPIE_UI_THEME

(Default: "blue")

Specifies the adjustable theme to apply Magpie UI pages. This theme consist principally of the applied color for generic interface items, but could be extended at a later date. The value must be one of the CSS file names located within the themes subdirectory.

Security Settings

Following configuration parameters are used to define specific values that are related to security configurations. Again, the Loading Settings will be processed beforehand and all magpie.[variable_name] setting definitions remain available as described at the start of the Configuration section.

MAGPIE_SECRET

Secret value employed to encrypt user authentication tokens.

Warning

Changing this value at a later time will cause previously created user tokens from passwords to be invalidated. This value MUST be defined before starting the application in order to move on to user accounts and permissions creation in your Magpie instance. The application will quit with an error if this value cannot be found.

Changed in version 2.0: Prior to this version, a default value was employed if this setting not provided. Later Magpie version now require an explicit definition of this parameter to avoid weak default configuration making the protected system prone to easier breaches. This also avoids incorrect initial setup of special :term:`User`s with that temporary weak secret that would need recreation to regenerate passwords.

MAGPIE_COOKIE_NAME

(Default: "auth_tkt")

Identifier of the cookie that will be used for reading and writing in the requests from login and for User authentication operations.

See also

Authentication and Authorization

MAGPIE_COOKIE_EXPIRE

[int] (Default: None)

Lifetime duration in seconds of the cookies. Tokens become invalid after this duration is elapsed.

When no value is provided, the cookies will have an infinite duration (never expire). When a valid integer value is provided, their reissue time (how long until a new token is regenerated) is a factor of 10 from this expiration time. For example, tokens are reissued after 360 seconds if their expiration is 3600.

MAGPIE_ADMIN_USER

Name of the default ‘administrator’ generated by the application.

See also

MAGPIE_ADMIN_PASSWORD

This User is required for initial launch of the application to avoid being ‘locked out’ as routes for creating new users require administrative access rights. It should be used as a first login method to setup other accounts. It is afterwards recommended to employ other user accounts with MAGPIE_ADMIN_GROUP membership to accomplish administrative management operations.

If this User is missing, it is automatically recreated on following application start. The best way to invalidate its credentials is therefore to completely remove its entry from the database so it gets regenerated from updated configuration values. Note also that modifying the value in the configuration without restarting the application so that the administrator user entry in the database can also be updated could cause other operations to fail drastically since this special user would be output of sync when employed by other Magpie operations such as Service Synchronization and Permission setup during the application startup.

Changed in version 2.0: Prior to this version, a default value was employed if this setting was not provided. Later Magpie version now require an explicit definition of this parameter to avoid weak default configuration making the protected system prone to easier breaches. This value MUST be defined before starting the application in order to resume to any other operation in your Magpie instance. The application will quit with an error if this value cannot be found. It is recommended that the developer configures every new instance with server-specific and strong credentials.

Prior versions also allowed modification of this value from the API and UI, which increased chances of having out-of-sync definitions between the database and Configuration files. This is not permitted anymore. Changing this value should be accomplished by updating the Configuration file and restarting the Web Application or calling the Magpie CLI Helpers to register changes.

MAGPIE_ADMIN_PASSWORD

Password of the default administrator User generated by the application.

See also

MAGPIE_ADMIN_USER

Changed in version 2.0: Default values definition and update during runtime for this parameter was modified to avoid problematic configuration synchronization problems. See corresponding change details in above MAGPIE_ADMIN_USER.

Changed in version 3.8: Prior to this version, changing only the MAGPIE_ADMIN_PASSWORD without modification of MAGPIE_ADMIN_USER was not handled. Following versions applies any password modification on restart to update credentials.

Warning

Note that if the password is modified in later versions, its new value will require to fulfill validation against standard password format requirements, such as MAGPIE_PASSWORD_MIN_LENGTH. Older passwords will remain effective only if left untouched for backward compatibility, but will be flagged as potential security risk.

MAGPIE_ADMIN_EMAIL

(Default: "${MAGPIE_ADMIN_USER}@mail.com")

Email of the default administrator generated by the application.

MAGPIE_ADMIN_GROUP

(Default: "administrators")

Name of the default administrator Group generated by the application.

Note

To simplify configuration of future administrators of the application, all their Inherited Permissions are shared through this Group instead of setting individual permissions on each User. It is recommended to keep defining such higher level permissions on this Group to ease the management process of granted access to all their members, or in other words, to allow multiple administrators to manage Magpie resources with their respective accounts.

MAGPIE_ADMIN_PERMISSION

[constant] (Value: "admin")

Name of the Permission used to represent highest administration privilege in the application. It is one of the special Access Permission known by the application (see also Route Access section).

MAGPIE_LOGGED_PERMISSION

[constant] (Value: "MAGPIE_LOGGED_USER")

Added in version 2.0.

Defines a special condition of Access Permission related to the Logged User session and the targeted User by the request. See details in Route Access for when it applies.

MAGPIE_LOGGED_USER

[constant] (Value: "current")

Keyword used to define route resolution using the currently Logged User. This value allows, for example, retrieving the user details of the logged user with GET /users/${MAGPIE_LOGGED_USER} instead of having to find explicitly the GET /users/<my-user-id> variant. User resolution is done using the authentication cookie found in the request. If no cookie can be found, it defaults to the MAGPIE_ANONYMOUS_USER value.

Note

Because the Logged User executing the request with this keyword is effectively the authenticated user, the behaviour of some specific paths can be slightly different than their literal user_name counterpart. For example, User details will be accessible to the Logged User (he can view his own information) but this same user will receive a forbidden response if using is ID in the path if he doesn’t have required privileges.

Changed in version 2.0: Even without administrative access rights, the Logged User is allowed to obtain some additional details about the targeted User of the request path if it corresponds to itself. See MAGPIE_LOGGED_PERMISSION and Route Access for further details.

MAGPIE_ANONYMOUS_USER

(Default: "anonymous")

Name of the default User that represents non logged-in user (ie: invalid or no Authentication token provided). This User is used to manage Public access to Service and Resource.

MAGPIE_ANONYMOUS_PASSWORD

[constant] (Value: ${MAGPIE_ANONYMOUS_USER})

Password of the default unauthenticated User. This value is not modifiable directly and is available only for preparation of the default user on startup.

MAGPIE_ANONYMOUS_EMAIL

(Default: "${MAGPIE_ANONYMOUS_USER}@mail.com")

Email of the default unauthenticated User.

MAGPIE_ANONYMOUS_GROUP

[constant] (Value: ${MAGPIE_ANONYMOUS_USER})

Special Group name that defines Public Access functionalities. All users are automatically member of this Public Group to obtain Inherited Permissions.

This parameter is enforced to be equal to MAGPIE_ANONYMOUS_USER. It is preserved for backward compatibility of migration scripts and external libraries that specifically refer to this parameter.

Changed in version 2.0: The Group generated by this configuration cannot be modified to remove User memberships or change other metadata associated to it.

Warning

To set Public permissions, one should always set them on this Group instead of directly on MAGPIE_ANONYMOUS_USER as setting them directly on that User will cause unexpected behaviours. See Public Access section for full explanation.

MAGPIE_EDITOR_GROUP

(Default: "editors")

Unused for the moment.

MAGPIE_USERS_GROUP

(Default: "users")

Name of a generic Group created to associate registered User memberships in the application.

Changed in version 2.0: New User are NOT automatically added to this Group anymore. This Group remains available for testing and backward compatibility reasons, but doesn’t have any special connotation and can be modified just as any other normal Group.

Prior versions of Magpie were adding every new User to that Group which made it no different than the behaviour fulfilled by MAGPIE_ANONYMOUS_GROUP which they are also member of. Since MAGPIE_USERS_GROUP has no special meaning and is modifiable at any time (e.g.: users could be removed from it), it could not even be employed to ensure provision of permissions applied to all users (its original purpose), which is also equivalent to functionalities provided with Public permissions inherited by MAGPIE_ANONYMOUS_GROUP that is more specifically handled by Magpie for this purpose.

MAGPIE_GROUP_NAME_MAX_LENGTH

[constant, int] (Value: 64)

Maximum length to consider a Group name as valid. Any name specified during creation will be forbidden if longer.

MAGPIE_USER_NAME_MAX_LENGTH

[constant, int] (Value: 64)

Maximum length to consider a User name as valid. Any name specified during creation will be forbidden if longer.

Warning

This value MUST NOT be greater than the token length used to identify a User to preserve internal functionalities.

MAGPIE_USER_NAME_EXTRA_REGEX

(Default: None)

Added in version 3.37.

A case sensitive (python3 syntax) regular expression used to validate a user_name when creating or updating a User.

For example, if MAGPIE_USER_NAME_EXTRA_REGEX='^\w+$', then a User can have userA as a user_name but not user.A or user-A.

Note that Magpie enforces other restrictions that must also be met for a user_name to be considered valid. This creates an additional restriction, it does not replace an existing restriction on the user_name.

If this variable is empty or unset, then no additional user_name validations will be performed.

MAGPIE_PASSWORD_MIN_LENGTH

[int] (Default: 12)

Added in version 2.0.

Minimum length of the password for User creation or update.

Note

For backward-compatibility requirements, passwords are not enforced this condition during login procedure as shorter passwords could have been used and not yet updated for older accounts. Fulfilling this requirement will be mandatory for new password updates and new User account creations.

MAGPIE_DEFAULT_PROVIDER

[constant] (Value: "ziggurat")

Name of the Provider used for login. This represents the identifier that is set to define how to differentiate between a local sign-in procedure and a dispatched one some known Authentication Providers.

Phoenix Settings

Following settings provide some integration support for Phoenix in order to synchronize its service definitions with Magpie services.

Warning

Support of Phoenix is fairly minimal. It is preserved for historical and backward compatibility but is not actively tested. Please submit an issue if you use it and some unexpected behaviour is encountered.

PHOENIX_USER

(Default: "phoenix")

Name of the user to use for Authentication in Phoenix.

PHOENIX_PASSWORD

(Default: "qwerty")

Password of the user to use for Authentication in Phoenix.

PHOENIX_HOST

(Default: ``${HOSTNAME}"``)

Hostname to use for Phoenix connection to accomplish Authentication and Service Synchronization.

PHOENIX_PORT

[int] (Default: 8443)

Port to use for Phoenix connection to accomplish Authentication and Service Synchronization.

PHOENIX_PUSH

[bool] (Default: True)

Whether to push new Service Synchronization settings to the referenced Phoenix connection.

Twitcher Settings

Following settings define parameters required by Twitcher (OWS Security Proxy) in order to interact with Magpie services as Policy Enforcement Point.

TWITCHER_PROTECTED_PATH

(Default: "/ows/proxy")

HTTP path used to define the protected (public) base path of services registered in Magpie that will be served by an existing Twitcher proxy application after Access Control List (ACL) verification of the Logged User.

Note

Using this parameter to define Twitcher’s path assumes that it resides under the same server domain as the Magpie instance being configured (ie: hostname is inferred from resolved value amongst MAGPIE_URL, MAGPIE_HOST, TWITCHER_HOST and HOSTNAME settings or environment variables).

Warning

Path is intended to be employed with Twitcher residing side-by-side with Magpie. Therefore, prefix /twitcher is added unless already explicitly provided. To employ another path without prefix, consider instead providing it with the full URL using TWITCHER_PROTECTED_URL parameter.

TWITCHER_HOST

(Default: None)

Added in version 2.0.

Specifies the explicit hostname to employ in combination with TWITCHER_PROTECTED_PATH to form the complete base service protected URL. Ignored if TWITCHER_PROTECTED_URL was provided directly. If not provided, hostname resolution falls back to using HOSTNAME environment variable.

Note

The resulting URL will take the form https://{TWITCHER_HOST}[/twitcher]{TWITCHER_PROTECTED_PATH} to imitate the resolution of TWITCHER_PROTECTED_URL considering provided TWITCHER_PROTECTED_PATH.

TWITCHER_PROTECTED_URL

(Default: see note)

Defines the protected (public) full base URL of services registered in Magpie. This setting is mainly to allow specifying an alternative domain where a remote Twitcher instance could reside.

Note

When not provided, attempts to infer the value by combining the environment variable HOSTNAME or TWITCHER_HOSTNAME, and an optional /twitcher prefix (as needed to match incoming request) and the value provided by TWITCHER_PROTECTED_PATH.

Please note that although Twitcher URL references are needed to configure interactive parameters with Magpie, the employed Twitcher instance will also need to have access to Magpie’s database in order to allow proper Service resolution with magpie.adapter.magpieservice.MagpieServiceStore. Appropriate database credentials must therefore be shared between the two services, as well as MAGPIE_SECRET value in order for successful completion of the handshake during Authentication procedure of the request User token.

Postgres Settings

Following settings define parameters required to define the PostgreSQL database connection employed by Magpie as well as some other database-related operation settings. Settings defined by magpie.[variable_name] definitions are available as described at the start of the Configuration section, as well as some special cases where additional configuration names are supported where mentioned.

MAGPIE_DB_MIGRATION

[bool] (Default: True)

Run database migration on startup in order to bring it up to date using Alembic.

MAGPIE_DB_MIGRATION_ATTEMPTS

[int] (Default: 5)

Number of attempts to re-run database migration on startup in case it failed (eg: due to connection error).

MAGPIE_DB_URL

(Default: see note)

Full database connection URL formatted as <db-type>://<user>:<password>@<host>:<port>/<db-name>.

Please refer to SQLAlchemy Engine’s documentation for supported database implementations and their corresponding configuration.

Warning

Only PostgreSQL has been extensively tested with Magpie, but other variants could be applicable, but will most likely than not require adjustments to support advanced operations handled by ziggurat_foundations. If another database implementation would better suit your needs, do not hesitate to open a `new issue`_ for potential PR integration.

Note

By default, postgresql database connection URL is inferred by combining following MAGPIE_POSTGRES_<> parameters if the value was not explicitly provided.

MAGPIE_POSTGRES_USERNAME

(Default: "magpie")

Database connection username to retrieve Magpie data stored in PostgreSQL.

Changed in version 1.9: On top of MAGPIE_POSTGRES_USERNAME, environment variable POSTGRES_USERNAME and setting postgres.username are all supported interchangeably. For backward compatibility, all above variants with user instead of username (with corresponding lower/upper case) are also verified for potential configuration if no prior parameter was matched. The lookup order of each name variant is as presented, while also keeping the setting name priority over an equivalent environment variable name.

MAGPIE_POSTGRES_PASSWORD

(Default: "qwerty")

Database connection password to retrieve Magpie data stored in PostgreSQL.

Changed in version 1.9: Environment variable POSTGRES_PASSWORD and setting postgres.password are also supported if not previously identified by their Magpie-prefixed variants.

MAGPIE_POSTGRES_HOST

(Default: "postgres")

Database connection host location to retrieve Magpie data stored in PostgreSQL.

Changed in version 1.9: Environment variable POSTGRES_HOST and setting postgres.host are also supported if not previously identified by their Magpie-prefixed variants.

MAGPIE_POSTGRES_PORT

[int] (Default: 5432)

Database connection port to retrieve Magpie data stored in PostgreSQL.

Changed in version 1.9: Environment variable POSTGRES_PORT and setting postgres.port are also supported if not previously identified by their Magpie-prefixed variants.

MAGPIE_POSTGRES_DB

(Default: "magpie")

Name of the database located at the specified connection to retrieve Magpie data stored in PostgreSQL.

Changed in version 1.9: Environment variable POSTGRES_DB and setting postgres.db, as well as the same variants with database instead of db, are also supported if not previously identified by their Magpie-prefixed variants.

GitHub Settings

To offer GitHub as an External Provider for authentication within Magpie, a trusted OAuth App or GitHub App must be created to grant necessary access to your Magpie instance to communicate with GitHub in order to retrieve profile Authentication tokens for users granting it the corresponding permission. You can customise the application name, image and description has deemed fit for your own Magpie instance.

Note

Both the OAuth App and GitHub App can be used interchangeably since the only access and scope required by Magpie is to validate GitHub user identities, which is supported by both approaches.

The Homepage URL used for configuring the Authentication application on GitHub should match the targeted Magpie instance URL. Other Application Settings values such as MAGPIE_URL, MAGPIE_HOST and HOSTNAME used to configure the instance must be considered. The Authorization callback URL should be defined to correspond to the provider signin endpoint, as defined below (with your relevant value resolution of MAGPIE_URL).

Homepage URL: ${MAGPIE_URL}
Callback URL: ${MAGPIE_URL}/providers/github/signin

Once created, variables GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET obtained from GitHub must be configured in the environment loaded by the Magpie instance. These settings correspond to the values that will be used to accomplish GitHub OAuth App - Web Application Flow.

During login on Magpie, users will then be able to select the GitHub External Provider using their GitHub usernames, as shown below.

Magpie UI login section using GitHub OAuth as External Provider.

The login page will perform a redirection toward the GitHub authentication page, asking them to login if necessary, and will ask to grant permissions to the OAuth application to access their GitHub identity for validation. After validation, the Callback URL will be used to return automatically to your Magpie instance, where the user should then be automatically logged in using the external identity. On future logins, it will not be necessary to re-validate the granted permissions as long as the OAuth application remains available and its GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET remain synchronized with the Magpie instance.

Warning

Note that, as during User creation, the same uniqueness rules apply for usernames and emails. Users will not be able to reuse the same username and/or email across internal and external Provider logins. If this poses issues for your use cases, contributions to Ouranosinc/Magpie#118 would be welcome to work on better supporting internal/external User associations.

See also

Refer to Authentication Requests and Authentication Providers for details.

WSO2 Settings

To use WSO2 authentication as External Provider, following variables must be set:

• WSO2_HOSTNAME

• WSO2_CLIENT_ID

• WSO2_CLIENT_SECRET

• WSO2_CERTIFICATE_FILE

• WSO2_SSL_VERIFY

To configure your Magpie instance as a trusted application for WSO2 (and therefore retrieve values of above parameters), please refer to WSO2 Identity Server Documentation.

See also

Refer to Authentication Requests and Authentication Providers for details.

User Registration and Approval Settings

Added in version 3.13.

This section describes the relevant details regarding the activation of settings MAGPIE_USER_REGISTRATION_ENABLED and MAGPIE_USER_REGISTRATION_APPROVAL_ENABLED. If those settings are not defined, or are explicitly set to False, all other options can be safely ignored.

Note

When any of the above parameters are enabled, the Application Settings regarding SMTP server and EMAIL options must also be defined.

All details regarding the procedures of registration and approval of user accounts are defined in section User Registration.

Following are the full description of all configuration parameters employed by the User registration and approval procedures.

MAGPIE_USER_REGISTRATION_ENABLED

[bool] (Default: False)

Added in version 3.13.

Specifies whether Magpie should provide User self-registration endpoints on /register/users for the API and /ui/register/users for the UI and enabled the registration procedure.

See also

See section User Registration for further details about this process.

When enabled, all other configuration regarding SMTP and EMAIL Application Settings must also be defined to properly send notification and validation email during registration.

The default value of this configuration setting is to preserve the original behavior of Magpie where no such User self-registration is possible. Therefore, the option must be explicitly defined to activate it.

Warning

Security Notice

Under normal operation (when disabled), Magpie can take advantage of stronger security by obfuscation as the user_name component is not accessible by any means other than administrator-level users. It is therefore hidden away from public view and acts as stronger credentials.

When this option is enabled, both the user_name and email of existing users become indirectly accessible for validation purposes, to avoid account conflicts during user registration. When enabling this option, the developer or server maintainer must be aware of these consideration.

For best security result, the setting should be activated only when the feature is required, and that user_name/email information is deemed adequate for potential public visibility, hence why the option is disabled by default. This is a design choice for respective servers and platforms.

MAGPIE_USER_REGISTRATION_SUBMISSION_EMAIL_TEMPLATE

(Default: magpie/api/templates/email_user_registration_submission.mako)

Added in version 3.13.

Path to a Mako Template file providing custom email format to send notification email to the Pending User following submission of a new User Registration.

When overridden with a custom email format, the contents should provide sufficient details indicating to the Pending User that its submitted email must be confirmed by visiting the link contained in that email. The confirmation URL would validate that emails can indeed be received by that Pending User to the submitted address be notified of future events.

The default template provides details about available template arguments.

MAGPIE_USER_REGISTRATION_NOTIFY_ENABLED

[bool] (Default: False)

Added in version 3.13.

Controls whether a notification email should be sent to MAGPIE_USER_REGISTRATION_NOTIFY_EMAIL_RECIPIENT once a Pending User successfully completed the registration process.

This can be used for example when no administrator validation is required (i.e.: MAGPIE_USER_REGISTRATION_APPROVAL_ENABLED is False), but that some platform manager still want to receive notices of any users that registered to its service.

Note

Enabling this option at the same time as MAGPIE_USER_REGISTRATION_APPROVAL_ENABLED while using the same email for both approval and notification could lead to noisy emails expeditions as approving administrators would be immediately notified of their own action of approving the user registration. Different emails can be set to communicate relevant notifications to intended parties. It is up to the developer to properly configure how verbose and to whom those emails should be addressed to.

MAGPIE_USER_REGISTRATION_NOTIFY_EMAIL_RECIPIENT

Added in version 3.13.

Email address where emails with contents defined by MAGPIE_USER_REGISTRATION_NOTIFY_EMAIL_TEMPLATE should be sent to when MAGPIE_USER_REGISTRATION_NOTIFY_ENABLED was activated.

MAGPIE_USER_REGISTRATION_NOTIFY_EMAIL_TEMPLATE

(Default: magpie/api/templates/email_user_registration_notify.mako)

Added in version 3.13.

Path to a Mako Template file providing custom email format to send notification email following completion of a new User Registration. The default template provides details about available template arguments.

A custom body must contain all relevant details defined in the default template to ensure basic functionalities of the User Registration workflow can be accomplished. The logic of the message content is left at the discretion of the developer if customized.

MAGPIE_USER_REGISTRATION_APPROVAL_ENABLED

[bool] (Default: False)

Added in version 3.13.

Specifies whether administrator approval is required to resume User Registration.

This setting is relevant only if MAGPIE_USER_REGISTRATION_ENABLED was also activated. When enabled and following email confirmation by the Pending User (see MAGPIE_USER_REGISTRATION_SUBMISSION_EMAIL_TEMPLATE), an email using following configuration options will be sent to notify the administrator authority that Pending User approval is awaiting their validation.

Approval process is bypassed if this setting is disabled, meaning that Pending User account will be immediately and automatically approved as soon as their email was validated, without any administrator intervention.

MAGPIE_USER_REGISTRATION_APPROVAL_EMAIL_RECIPIENT

Added in version 3.13.

Email of the administrator to which a notification is sent using the body defined by MAGPIE_USER_REGISTRATION_APPROVAL_EMAIL_TEMPLATE, when a new user registration was requested.

The email employed for this parameter can be toward any target, including an email that does not correspond to any User in the Magpie database. For example, that email could be for a shared user support team that replies to those requests. Note that to validate the user registration though, valid administrative-level User with matching credentials will be required to complete the process.

MAGPIE_USER_REGISTRATION_APPROVAL_EMAIL_TEMPLATE

(Default: magpie/api/templates/email_user_registration_approval.mako)

Added in version 3.13.

Path to a Mako Template file providing custom email format to send notification email to MAGPIE_USER_REGISTRATION_APPROVAL_EMAIL_RECIPIENT following a submitted user registration that must be approved or declined by the administrator.

When overridden with a custom email format, the contents should provide sufficient details indicating to the administrator which Pending User requested a new account registration, and links where it can review it to be approved or declined.

The default template provides details about available template arguments.

MAGPIE_USER_REGISTRATION_APPROVED_EMAIL_TEMPLATE

(Default: magpie/api/templates/email_user_registration_approved.mako)

Added in version 3.13.

Path to a Mako Template file providing custom email format to send an email to the Pending User that initially submitted the user registration to notify them that the registration process was successfully approved and completed, and that their account is active starting from that moment.

The default template provides details about available template arguments.

Note

This email template is employed regardless of value defined for setting MAGPIE_USER_REGISTRATION_APPROVAL_ENABLED. When administrator approval is enabled, the email will be sent only after the account was approved. Otherwise, it is sent as soon as email conformation is obtained from the Pending User. Parameter approval_required is provided to generate alternative Mako Template contents in case different messages should be sent for each situation.

MAGPIE_USER_REGISTRATION_DECLINED_EMAIL_TEMPLATE

(Default: magpie/api/templates/email_user_registration_declined.mako)

Added in version 3.13.

Path to a Mako Template file providing custom email format to send an email to the Pending User that initially submitted the user registration to notify them of that their user registration request was declined by the administrator following approval process.

The default template provides details about available template arguments.

User-Group Assignment Settings

Added in version 3.17.

Following are the full description of all configuration parameters employed by the User-Group assignment procedures, in the case of a Group that requires terms and conditions validation by the User.

MAGPIE_GROUP_TERMS_SUBMISSION_EMAIL_TEMPLATE

(Default: magpie/api/templates/email_group_terms_submission.mako)

Added in version 3.17.

Path to a Mako Template file providing custom email format to send notification email to the User following submission of the User assignment to a Group that requires accepting terms and conditions.

When overridden with a custom email format, the contents should provide sufficient details indicating to the User that they must accept the Group’s terms and conditions to join it, and that confirmation is accomplished by visiting the link contained in that email. The confirmation URL would validate that the User accepts the terms and conditions, and would proceed with the assignment of the User to the Group. The contents of the email should also include the terms and conditions of the Group.

The default template provides details about available template arguments.

MAGPIE_GROUP_TERMS_APPROVED_EMAIL_TEMPLATE

(Default: magpie/api/templates/email_group_terms_approved.mako)

Added in version 3.17.

Path to a Mako Template file providing custom email format to send an email to the User related to a User-Group assignment to notify them that the terms and conditions were accepted, and that their account is now a member of the requested Group.

The default template provides details about available template arguments.

Webhook Configuration

Added in version 3.6: The concept of Webhook is introduced only following this version, and further improved in following ones.

A webhooks section can be added to the Combined Configuration File. This section defines a list of URLs and request parameters that should be called following or during specific events, such as but not limited to, creating or deleting a User.

Note

Webhook requests are asynchronous, so Magpie might execute other requests before the webhooks requests are completed and processed.

See also

See Combined Configuration File for a minimal example of where and how to define the webhooks section.

Each Webhook implementation provides different sets of parameters according to its action. Those parameters can be employed to fill a template request payload defined under payload. See magpie.api.webhook.WebhookAction and below sub-sections for supported values.

To register any Webhook to be called at runtime upon corresponding events, following parameters must be defined. Configuration parameters are all required unless explicitly indicated to have a default value.

• name

  The name of the Webhook for reference.

  It is not required for this name to be unique, but it is recommended for reporting and reference purposes. If duplicates are found, a warning will be emitted, but all entries will still be registered.

• action

  (Values: one of magpie.api.webhook.WebhookAction)

  The action event defining when the corresponding Webhook must be triggered for execution.

  See also

  Webhook Actions for details about each implementation.

• method

  (Values: one of magpie.api.webhook.WEBHOOK_HTTP_METHODS)

  The HTTP method used for the Webhook request.

• url

  A valid HTTP(S) URL location where the triggered Webhook request will be sent.

• format

  (Default: "json", Value: one of magpie.utils.FORMAT_TYPE_MAPPING)

  A valid format definition of the content type of payload.

• payload

  (Default: None)

  Structure of the payload that will be sent in the request body of the triggered Webhook. The payload can be anything between a literal string or a JSON/YAML formatted structure.

  Changed in version 3.12: If the field is undefined or resolved as None, it will be accepted for request with an empty body.

  Note

  The payload can employ parameters that contain template variables using brace characters {{<variable>}}. Applicable {{<variable>}} substitution are respective to each webhook action, as presented in Webhook Actions.

  See also

  See Webhook Template Payload for a more concrete example of templated payload definition.

Webhook Actions

This section presents the supported Webhook action values that can be registered and corresponding template parameters available in each case to generate the payload.

User Creation

Action	WebhookAction.CREATE_USER
Parameters	{{user.name}}, {{user.id}}, {{user.email}}, {{callback_url}}

Triggered whenever a User gets successfully created, using a POST /users request.

The User details are provided for reference as needed for the receiving external web application defined by the configured url.

The callback_url serves as follow-up endpoint, should the registered external application need it, to request using HTTP GET method (no body) that Magpie sets the User account status as erroneous. That User would then be affected with status value magpie.models.UserStatuses.WebhookError. The callback_url location will be available until called or expired according to MAGPIE_TOKEN_EXPIRE setting. When no request is sent to the callback_url, the created User is assumed valid and its account is attributed magpie.models.UserStatuses.OK status.

User Deletion

Action	WebhookAction.DELETE_USER
Parameters	{{user.name}}, {{user.id}}, {{user.email}}

Triggered whenever a User gets successfully deleted, using a DELETE /users/{user_name} request.

User Status Update

Action	WebhookAction.UPDATE_USER_STATUS
Parameters	{{user.name}}, {{user.id}}, {{user.status}}, {{callback_url}}

Triggered whenever a User status gets successfully updated, using a PATCH /users/{user_name} request.

This event DOES NOT apply to changes of User status caused by callback URL request received following a User Creation event.

The callback_url in this case can be requested with GET method (no body) to ask Magpie to reset the just updated User account status to magpie.models.UserStatuses.WebhookError. This Webhook can be employed to retry an external operation of the registered application, by triggering status updates, and only consider the complete operation successful when no further callback_url requests are received.

Permission Updates

Below Webhook implementations can all be configured for any combination of creation/deletion of a Permission for a User or Group, and targeting either a Service or a Resource.

Action	WebhookAction.CREATE_USER_PERMISSION, WebhookAction.DELETE_USER_PERMISSION, WebhookAction.CREATE_GROUP_PERMISSION, WebhookAction.DELETE_GROUP_PERMISSION
Parameters	{{user.name}} or {{group.name}}, {{user.id}} or {{group.id}}, {{resource.id}}, {{resource.type}}, {{resource.name}}, {{resource.display_name}}, {{service.name}}, {{service.type}}, {{service.public_url}}, {{service.sync_type}}, {{permission.name}}, {{permission.access}}, {{permission.scope}}, {{permission}}

The parameters available for the payload are very similar in each case, except that they are adjusted accordingly to the User or Group the modification applies to.

The Resource details are available regardless of if it refers to a Service or any children Resource. The value of {{resource.type}} will be "service" if the reference was a Service. The {{service.<field>}} parameters will only be defined if the target was indeed a Service, and will be null otherwise.

The created or deleted Permission details are available with different formats. The {{permission.name}}, {{permission.access}} and {{permission.scope}} correspond to the same fields presented in Permission Definition and Modifiers chapter. The permission parameter corresponds to the explicit name, as defined in Permissions Representation.

Webhook Template Payload

Following subsections demonstrate common substitution patterns for templated request payload according to desired content format.

JSON Payload

This is a minimal example to demonstrate how the Webhook template payload functionality can help customize requests sent following given event triggers. For simplicity, lets assume that a demo Webhook provides two parameters, namely user_name = "demo" and user_id = 123. Let’s assume the following configuration was defined and loaded by Magpie.

webhooks:
  - name: demo_webhook
    action: demo
    method: POST
    url: https://receiving-middleware.example.com
    payload:
      user:
        name: "{{user.name}}"
        id: "{{user.id}}"
        str: "'{{user.id}}'"
      msg: Hello {{user.name}}, your ID is {{user.id}}

Upon trigger of the demo event, the above Webhook definition would result in a request sent with the following JSON payload contents.

{
    "user": {
        "name": "demo",
        "id": 123,
        "str": "123"
    },
    "msg": "Hello demo, your ID is 123"
}

As presented above, the "{{user.name}}" from the template gets substituted by the corresponding "demo" value. Similarly, "{{user.id}}" is replaced by 123. An important thing to notice is that value types are preserved, which is why the id field is an integer since that corresponding parameter is an integer in Magpie. Using the specification "'{{user.id}}'" (with additional single quotes) instead tells the template parser to replace the value by its string representation. It is also possible to define any combination of parameters as indicated in the msg field of the example, and for any kind of structure, as long as JSON/YAML valid definitions are respected.

It is important to take into consideration how YAML parsing operates in this case. Quotes are not mandatory everywhere, such as for values where inferred type is guaranteed to be strings as for msg, but this can rapidly become a problem in other cases such as within an object definition or for field keys. It is therefore recommended to employ quotes whenever possible to remove ambiguity.

Another example where YAML parsing must be carefully considered is as in the following definition that could produce an unexpected outcome.

payload:
  user:
    - {user_name}

This would generate the following JSON content.

{
    "user": {
        "user_name": null
    }
}

This is because YAML interprets {user_name} within an array list as an object with a field named user_name and no corresponding value (i.e.: null). For this reason, Magpie employs the double-braced {{<variable>}} format to remove this ambiguity. An unknown parameter value defined in payload during substitution or an ill defined configuration at application startup would immediately generate an error since YAML parsing will not correctly understand nor be able to infer the format of the double-braces definitions, instead of silently failing. When using a parameter by themselves, such as in the top example’s "{{user.name}}" and "{{user.id}}" values, quotes will usually be required.

String Payload

Literal string body can also be employed using templated payload definition to form a custom Webhook request content format. To do so, one only needs to define the payload as a string. For convenience, multiline character (e.g.: |) can be employed to ease literal formatting of the content as in the below example.

payload: |
  param: {{user.name}}
  quote: "{{user.id}}"

This would produce the literal string output as below.

param: demo
quote: "123"

It is important to consider that in this case, because the whole payload is a string, explicit quotes and newlines defined in its value will remain as is, according to the selected multiline character. Also, this kind of Webhook should most probably define the appropriate format value if the default json is not the desired Content-Type, as Magpie will not attempt to infer the content structure to generate the request.

Advanced Payload Substitutions

An extensive representation of supported template replacement patterns is presented in the following tests.test_webhooks.test_webhook_template_substitution() function. As presented, the resulting payload can therefore be extensively customized to match exactly the desired format.

def test_webhook_template_substitution():
    """
    Verify that webhook template replacement works as expected against parameter values of different types.

    For example, a list item or dictionary value can be an integer, which should be preserved.
    If quotes are added, the non-string fields should then be converted to string as expected.
    Quotes on string fields though are redundant and should be ignored.
    Additional repeated quotes should leave them as specified.
    """
    params = {"user.name": "test", "user.id": 123}
    spec = yaml.safe_load(inspect.cleandoc("""
    payload:
      param: 
        name: "{{user.name}}"
        id: "{{user.id}}"
        id_str: "'{{user.id}}'"
        none: user.id               # only plain name, not a template substitution
        str: "{user.name}"          # literal field with '{user.name}', not a template substitution
        obj: {user.name}            # object with field 'user.name', not a template substitution
      compose:
        id: user_{{user.id}}
        msg: Hello {{user.name}}, your ID is {{user.id}}
      key_str: 
        "{{user.id}}": "id"
        "{{user.name}}": "name"
      listed: 
        - "{{user.id}}"
        - "{{user.name}}"
        - "'{{user.id}}'"
      quoted: 
        explicit: "{{user.name}}"
        single: "\'{{user.name}}\'"
        double: "\\"{{user.name}}\\""
        multi: "\\"\'\'\\"{{user.name}}\\"\'\'\\""
    """))
    expect = {
        "param": {
            "name": params["user.name"],
            "id": params["user.id"],
            "id_str": str(params["user.id"]),
            "str": "{user.name}",
            "obj": {"user.name": None},  # format is not a template, but a valid YAML definition
            "none": "user.id"  # was not a template, remains literal string not replaced by value
        },
        "compose": {
            "id": "user_{}".format(params["user.id"]),
            "msg": "Hello {}, your ID is {}".format(params["user.name"], params["user.id"])
        },
        "key_str": {
            str(params["user.id"]): "id",
            params["user.name"]: "name"
        },
        "listed": [
            params["user.id"],
            params["user.name"],
            str(params["user.id"])
        ],
        "quoted": {
            "explicit": "{}".format(params["user.name"]),
            "single": "'{}'".format(params["user.name"]),
            "double": "\"{}\"".format(params["user.name"]),
            "multi": "\"\'\'\"{}\"\'\'\"".format(params["user.name"])
        }
    }
    data = utils.check_no_raise(lambda: replace_template(params, spec["payload"]))
    utils.check_val_equal(data, expect, diff=True)