Protocol

UMC uses its own protocol for internal communication that is optimized for the needs of UMC. The following documents the specification of the protocol and the message format as well as the API of the Python implementation.

Specification

This new generation of UMCP is based on the version 1.0 but is not compatible.

This protocol is used by the UMC server for external clients and between the UMC server and its UMC module processes.

Data flow

The protocol is based on a server/client model. The client sends requests to the server that will be answered with a response message by the server.

With a status code in the response message the client can determine the type of result of its request:

• An error occurred during the processing of the request. The status

  code contains details of the error

• The command was processed successfully. A status message may contain

  details about the performed task

Authentication

Before a client may send request messages to the server that contain commands to execute, the client has to authenticate. After a successful authentication the UMC server determines the permissions for the user defined by policies in the LDAP directory. If the LDAP server is not reachable a local cache is checked for previously discovered permissions. If none of these sources is available the user is prohibited to use any command.

The authentication process within the UMC server uses the PAM service univention-management-console. By default, this service uses a cache for credentials if the LDAP server is not available to provide the possibility to access the UMC server also in case of problems with the LDAP server.

Message format

The messages, request and response, have the same format that consists of a single header line, one empty line and the body.

The header line contains control information that allows the UMC server to verify the correctness of the message without reading the rest of the message.

Message header

The header defines the message type, a unique identifier, the length of the message body in bytes, the command and the MIME type of the body.

(REQUEST|RESPONSE)/<id>/<length of body>[/<mime-type>]: <command>[ <arguments>]

The message contains the following parts:

MESSAGE TYPE

By the first keyword the message type is defined. Supported message types are REQUEST and RESPONSE. Any other type will be ignored.

ID

Separated by a / the message id follows that must be unique within a communication channel. By default it consists of a timestamp and a counter.

LENGTH

The next field is a number defining the length of the body in bytes. Starting to count after the empty line.

MIME TYPE

Since UMCP 2.0 there is as another field specifying the MIME type of the body. If not given the guessed value for the MIME type is application/json. If the body can not be decoded using a JSON parser the message is invalid.

COMMAND

The last two fields define the UMCP command that should be executed by the server. The following commands are supported:

AUTH

sends an authentication request. It must be the first command send by the client. All commands send before a successful authentication are rejected. This is no longer true and the AUTH command can be replaced by a SET command to set information about the user and other details.

GET

is used to retrieve information from the UMC server, e.g. a list of all UMC modules available in this session.

SET

is used to define settings for the session, e.g. the language.

COMMAND

This command is used to pass requests to UMC modules. Each module defines a set of commands that it implements. The UMC module command is defined by the first argument in the UMCP header, e.g. a request like

REQUEST/123423423-01/42/application/json: COMMAND ucr/query

passes on the module command ucr/query to a UMC module.

Message body

The message body may contain one object of any type, e.g. an image, an Open Office document or JSON, which is the default type and is the only supported MIME type for REQUEST messages. It contains a dictionary that has a few pre-defined keys (for both message types):

options

contains the arguments for the command.

status

defines the status code in response messages. The codes are similar to the HTTP status codes , e.g. 200 defines a successful execution of the command. The appendix contains a detailed list [[#Status-Codes]].

message

may contain a human readable description of the status code. This may contain details to explain the user the situation.

flavor

is an optional field. If given in a request message the module may act differently than without the flavor.

Examples

This section contains a few example messages of UMCP 2.0

Authentication request

REQUEST/130928961341733-1/47/application/json: AUTH
{"username": "root", "password": "univention"}

REQUEST/130928961341733-2/705/application/json: SET
{"options":{"locale":"de_DE.UTF-8","credentials":{"auth_type":null,"user_dn":"uid=Administrator,cn=users,dc=test,dc=local","username":"Administrator","password":"univention"},"commands":{"description":"Overview of processes on the local system","translationId":"","keywords":["process","Process overview"],"id":"top","categories":["system"],"icon":null,"flavors":[],"commands":[{"allow_anonymous":false,"method":"query","name":"top/query"},{"allow_anonymous":false,"method":"kill","name":"top/kill"}],"name":"Process overview","url":null,"priority":50.0,"required_commands":[],"version":"8.0.1-3A~4.4.0.201910141206"},"acls":[{"fromUser":false,"flavor":"*","command":"top/*","host":"dc0","options":{}}]}}

Request: Run top query

REQUEST/130928961341733-3/16/application/json: COMMAND top/query
{"options": {}}

Request: Search for users

REQUEST/130928961341726-0/125/application/json: COMMAND udm/query
{"flavor": "users/user", "options": {"objectProperty": "name", "objectPropertyValue": "test1*1", "objectType": "users/user"}}

Response: Search for users

RESPONSE/130928961341726-0/1639/application/json: COMMAND udm/query
{"status": 200, "message": null, "options": {"objectProperty": "name", "objectPropertyValue": "test1*1", "objectType": "users/user"}, "result": [{"ldap-dn": "uid=test11,cn=users,dc=univention,dc=qa", "path": "univention.qa:/users", "name": "test11", "objectType": "users/user"}, {"ldap-dn": "uid=test101,cn=users,dc=univention,dc=qa", "path": "univention.qa:/users", "name": "test101", "objectType": "users/user"}, {"ldap-dn": "uid=test111,cn=users,dc=univention,dc=qa", "path": "univention.qa:/users", "name": "test111", "objectType": "users/user"}, {"ldap-dn": "uid=test121,cn=users,dc=univention,dc=qa", "path": "univention.qa:/users", "name": "test121", "objectType": "users/user"}, {"ldap-dn": "uid=test131,cn=users,dc=univention,dc=qa", "path": "univention.qa:/users", "name": "test131", "objectType": "users/user"}, {"ldap-dn": "uid=test141,cn=users,dc=univention,dc=qa", "path": "univention.qa:/users", "name": "test141", "objectType": "users/user"}, {"ldap-dn": "uid=test151,cn=users,dc=univention,dc=qa", "path": "univention.qa:/users", "name": "test151", "objectType": "users/user"}, {"ldap-dn": "uid=test161,cn=users,dc=univention,dc=qa", "path": "univention.qa:/users", "name": "test161", "objectType": "users/user"}, {"ldap-dn": "uid=test171,cn=users,dc=univention,dc=qa", "path": "univention.qa:/users", "name": "test171", "objectType": "users/user"}, {"ldap-dn": "uid=test181,cn=users,dc=univention,dc=qa", "path": "univention.qa:/users", "name": "test181", "objectType": "users/user"}, {"ldap-dn": "uid=test191,cn=users,dc=univention,dc=qa", "path": "univention.qa:/users", "name": "test191", "objectType": "users/user"}]}

Python API

The following documents the API of all components implementing parts of the Univention Management Concole Protocol (UMCP). The figure Core structure of the UMC implementation shows the relations between the classes implementing the UMC service.

Core structure of the UMC implementation

Messages

UMCP is a simple RPC protocol using two message types (request and response message). The API of the Python objects representing the messages are based on the class Message.

exception univention.management.console.protocol.message.IncompleteMessageError

Bases: exceptions.Exception

class univention.management.console.protocol.message.Message(type=1, command=u'', mime_type='application/json', data=None, arguments=None, options=None)

Bases: object

Represents a protocol message of UMCP. It is able to parse request as well as response messages.

Parameters:

• type – message type (RESPONSE or REQUEST)
• command (str) – UMCP command
• mime_type (str) – defines the MIME type of the message body
• data – binary data that should contain a message
• arguments – arguments for the UMCP command
• options – options passed to the command handler. This works for request messages with MIME type application/JSON only.

REQUEST = 1

RESPONSE = 0

_Message__counter = 0

_create_id()

static _formattedMessage(_id, _type, mimetype, command, body, arguments)

Returns formatted message.

_get_key(key, default=None)

_header = <_sre.SRE_Pattern object>

_set_key(key, value, cast=None)

cookies

error

flavor

headers

http_method

id

is_type(type)

Checks the message type

message

options

parse(msg)

Parses data and creates in case of a valid UMCP message the corresponding object. If the data contains more than the message the rest of the data is returned.

Raises:ParseError

reason

recreate_id()

Creates a new unique ID for the message

result

status

exception univention.management.console.protocol.message.ParseError

Bases: exceptions.Exception

class univention.management.console.protocol.message.Request(command, arguments=None, options=None, mime_type='application/json')

Bases: univention.management.console.protocol.message.Message

Represents an UMCP request message

class univention.management.console.protocol.message.Response(request=None, data=None, mime_type='application/json')

Bases: univention.management.console.protocol.message.Message

This class describes a response to a request from the console frontend to the console daemon

recreate_id = None

set_body(filename, mimetype=None)

Set body of response by guessing the mime type of the given file if not specified and adding the content of the file to the body. The mime type is guessed using the extension of the filename.

Server, client and session

Defines the basic class for an UMC server.

class univention.management.console.protocol.server.MagicBucket

Bases: object

Manages a connection (session) to the UMC server. Therefore it ensures that without successful authentication no other command is accepted. All commands are passed to the SessionHandler. After the user has authenticated the commands are passed on to the Processor.

new(client, socket)

Is called by the Server object to announce a new incoming connection.

Parameters:

• client (str) – IP address + port
• socket (fd) – a file descriptor or socket object

reset_connection_timeout(state)

_timed_out(state)

Closes the connection after a specified timeout

exit()

Closes all open connections.

_receive(sock)

Signal callback: Handles incoming data. Processes SSL events and parses the incoming data. If a valid UMCP was found it is passed to _handle.

Parameters:socket (fd) – file descriptor or socket object that reported incoming data

_do_send(sock)

_response(msg, state)

Send UMCP response to client. If the status code is 250 the module process is asking for exit. This method forfills the request.

_cleanup(socket)

class univention.management.console.protocol.server.Server(port=6670, ssl=True, unix=None, magic=True, magicClass=<class 'univention.management.console.protocol.server.MagicBucket'>, load_ressources=True, processes=1)

Bases: notifier.signals.Provider

Creates an UMC server. It handles incoming connections on UNIX or TCP sockets and passes the control to an external session handler (e.g. MagicBucket)

Parameters:

• port (int) – port to listen to
• ssl (bool) – if SSL should be used
• unix (str) – if given it must be the filename of the UNIX socket to use
• magic (bool) – if an external session handler should be used
• magicClass (class) – a reference to the class for the external session handler
• load_ressources (bool) – if the modules and categories definitions should be loaded

Initializes the socket to listen for requests

_connection(socket)

Signal callback: Invoked on incoming connections.

exit()

Shuts down all open connections.

static reload()

Reloads resources like module and category definitions

static analyse_memory()

Print the number of living UMC objects. Helpful when analysing memory leaks.

_Server__verify_cert_cb(conn, cert, errnum, depth, ok)

class univention.management.console.protocol.server.State(client, socket)

Bases: object

Holds information about the state of an active session

Parameters:

• client (str) – IP address + port
• socket (fd) – file descriptor or socket object

client

socket

buffer

requests

resend_queue

session

_timer

reset_connection_timeout()

active

timeout

Provides a class Client that implements an UMCP client

exception univention.management.console.protocol.client.UnknownRequestError

Bases: exceptions.Exception

exception univention.management.console.protocol.client.NoSocketError

Bases: exceptions.Exception

exception univention.management.console.protocol.client.ConnectionError

Bases: exceptions.Exception

class univention.management.console.protocol.client.Client(servername='localhost', port=6670, unix=None, ssl=True)

Bases: notifier.signals.Provider, univention.lib.i18n.Translation

Implements an UMCP client

Parameters:

• servername (str) – hostname of the UMC server to connect to
• port (int) – port number of the UMC server
• unix (str) – filename of the UNIX socket to connect to
• ssl (bool) – if True the connection is encrypted
• auth (bool) – if False no authentication is required for the connection

Initialize a socket-connection to the server.

openRequests

Returns a list of open UMCP requests

_init_socket()

disconnect(force=True)

Shutdown the connection. If there are still open requests and force is False the connection is kept.

connect()

Connects to the UMC server

_resend(sock)

request(msg)

Sends an UMCP request to the UMC server

Parameters:msg (Request) – the UMCP request to send

invalidate_all_requests(status=500, message=None)

Checks for open UMCP requests and invalidates these by faking a response with the given status code

_recv(sock)

_handle(response)

authenticate(msg)

Authenticate against the UMC server

_Client__closed()

_Client__reconnect_without_ssl()

_Client__verify_cert_cb(conn, cert, errnum, depth, ok)

This module provides a class for an UMC module server. it is based on the UMC server class Server.

class univention.management.console.protocol.modserver.ModuleServer(socket, module, timeout=300, check_acls=True)

Bases: univention.management.console.protocol.server.Server

Implements an UMC module server

Parameters:

• socket (str) – UNIX socket filename
• module (str) – name of the UMC module to serve
• timeout (int) – If there are no incoming requests for timeout seconds the module server shuts down
• check_acls (bool) – if False the module server does not check the permissions (dangerous!)

_load_module()

_reply(msg, final)

_timer()

In order to avoid problems when the system time is changed (e.g., via rdate), we register a timer event that counts down the session timeout second-wise.

_timed_out()

_client(client, socket)

_recv(sock)

error_handling(request, method, etype, exc, etraceback)

handle(msg)

Handles incoming UMCP requests. This function is called only when it is a valid UMCP request.

Parameters:msg (Request) – the received UMCP request

The following commands are handled directly and are not passed to the custom module code:

• SET (acls|username|credentials)
• EXIT

command_get(command_name)

Returns the command object that matches the given command name

command_is_known(command_name)

Checks if a command with the given command name is known

Return type:bool

_do_send(sock)

response(msg)

Sends an UMCP response to the client

Implements several helper classes to handle the state of a session and the communication with the module processes

class univention.management.console.protocol.session.ModuleProcess(module, debug='0', locale=None)

Bases: univention.management.console.protocol.client.Client

handles the communication with a UMC module process

Parameters:

• module (str) – name of the module to start
• debug (str) – debug level as a string
• locale (str) – locale to use for the module process

stop()

_died(pid, status)

_response(msg)

pid()

Returns process ID of module process

class univention.management.console.protocol.session.ProcessorBase

Bases: univention.management.console.base.Base

Implements a proxy and command handler. It handles all internal UMCP commands and passes the commands for a module to the subprocess.

Parameters:

• username (str) – name of the user who authenticated for this session
• password (str) – password of the user

lo

set_locale(locale)

set_credentials(username, password, auth_type)

_reload_acls_and_permitted_commands()

_reload_acls()

_reload_i18n()

_search_user_dn()

error_handling(etype, exc, etraceback)

request(msg)

Handles an incoming UMCP request and passes the requests to specific handler functions.

Parameters:msg (Request) – UMCP request

handle_request_unknown(msg)

Handles an unknown or invalid request

handle_request_unauthorized(msg)

handle_request_auth(request)

handle_request_get_ucr(msg)

handle_request_get_info(msg)

handle_request_get_user_preferences(msg)

handle_request_get_hosts(msg)

handle_request_set_password(msg)

handle_request_set_user(msg)

handle_request_version(msg)

handle_request_get(msg)

Handles a GET request

META_JSON_PATH = '/var/www/univention/meta.json'

handle_request_get_meta(request)

handle_request_set(msg)

handle_request_get_modules(request)

_get_user_favorites()

handle_request_get_categories(request)

handle_request_set_locale(request)

update_module_passwords()

handle_request_upload(request)

Handles an UPLOAD request. The command is used for the HTTP access to the UMC server. Incoming HTTP requests that send a list of files are passed on to the UMC server by storing the files in temporary files and passing the information about the files to the UMC server in the options of the request. The request options must be a list of dictionaries. Each dictionary must contain the following keys:

• filename – the original name of the file
• name – name of the form field
• tmpfile – filename of the temporary file

Parameters:msg (Request) – UMCP request

handle_request_command(msg)

Handles a COMMAND request. The request must contain a valid and known command that can be accessed by the current user. If access to the command is prohibited the request is answered as a forbidden command.

If there is no running module process for the given command a new one is started and the request is added to a queue of requests that will be passed on when the process is ready.

If a module process is already running the request is passed on and the inactivity timer is reset.

Parameters:msg (Request) – UMCP request

_mod_connect(mod, msg)

Callback for a timer event: Trying to connect to newly started module process

_mod_inactive(module)

_socket_died(module_name)

_mod_error(exc, module_name)

_mod_died(pid, status, module_name)

reset_inactivity_timer(module)

Resets the inactivity timer. This timer watches the inactivity of the module process. If the module did not receive a request for MODULE_INACTIVITY_TIMER seconds the module process is shut down to save resources. The timer ticks each seconds to handle glitches of the system clock.

Parameters:module (Module) – a module

_inactivitiy_tick(module)

handle_request_exit(msg)

Handles an EXIT request. If the request does not have an argument that contains a valid name of a running UMC module instance the request is returned as a bad request.

If the request is valid it is passed on to the module process. Additionally a timer of 3000 milliseconds is started. After that amount of time the module process MUST have been exited itself. If not the UMC server will kill the module process.

Parameters:msg (Request) – UMCP request

_purge_child(module_name)

shutdown()

Instructs the module process to shutdown

class univention.management.console.protocol.session.Processor

Bases: univention.management.console.protocol.session.ProcessorBase

handle_request_get_ucr(request)

META_UCR_VARS = ['domainname', 'hostname', 'ldap/master', 'license/base', 'server/role', 'ssl/validity/host', 'ssl/validity/root', 'ssl/validity/warning', 'umc/web/favorites/default', 'umc/web/piwik', 'update/available', 'update/reboot/required', 'uuid/license', 'uuid/system', 'version/erratalevel', 'version/patchlevel', 'version/releasename', 'version/version']

handle_request_get_meta(request)

CHANGELOG_VERSION = <_sre.SRE_Pattern object>

handle_request_get_info(request)

handle_request_get_hosts(request)

handle_request_set_password(request)

_password_changed(thread, result, request, new_password)

handle_request_get_user_preferences(request)

handle_request_set_user(request)

_get_user_preferences(lo)

_set_user_preferences(lo, preferences)

handle_request_version(msg)

Handles a VERSION request by returning the version of the UMC server’s protocol version.

Parameters:msg (Request) – UMCP request

class univention.management.console.protocol.session.SessionHandler

Bases: univention.management.console.protocol.session.ProcessorBase

has_active_module_processes()

_reload_acls()

All unauthenticated requests are passed here. We need to set empty ACL’s

error_handling(etype, exc, etraceback)

_authentication_finished(result, request)

_authentication_finished2(request, result)

handle(request)

Ensures that commands are only passed to the processor if a successful authentication has been completed.

_handle_auth(request)

initalize_processor(request)

_response(response)

parse_error(request, parse_error)

close_session()