Validation tool for Murano Application Packages¶
https://blueprints.launchpad.net/murano/+spec/murano-app-validation
Murano doesn’t have validation mechanisms for MuranoPL language and application packages.
This spec proposes a validation toolkit that improves workflow by making error detection possible at the early stages of application development lifecycle.
Problem description¶
Early error detection simplifies application development, troubleshooting and support.
Currently Murano does not provide mechanisms to perform validation of the Murano application package before running deployment process. Thus Murano cannot prevent uploading of invalid package and developer cannot verify that application package contains valid structure and code before deployment process.
Many problems with application packages (such as invalid package structure, errors in code, typos, etc.) can be detected and solved by using static validation tool.
Proposed change¶
Introduce a new toolkit for validation MuranoPL language and Murano application packages.
- Validation toolkit SHOULD be implemented as a standalone Python package (library) with CLI interface.
- Validation library will be used to validate Murano packages during loading process.
- CLI tool can be used by application developer to perform early checks before package is assembled.
- Toolkit should be extensible with custom validators. It will provide API for extension purposes.
- Toolkit should be configurable to run list of specified inspections or to ignore specified inspections.
Toolkit should implement the following validation suites: * Manifest validation * Classes validation * UI definition validation Other validators will be implemented later as part of this tool or as external packages
Manifest validation¶
- manifest.yaml file should exist and should be a valid YAML document.
- manifest.yaml structure should respond to manifest structure described in Manifest structure.
- All files referenced in Classes section should exist in Classes subdirectory and should be valid YAML documents.
- Warning should be reported in case when additional files exist in Classes subdirectory but not referenced in the manifest file.
- UI definition file should exist for Application package (MuranoPL =< 1.4)
Class validation¶
- Valid classes and methods should check if properties are defined correctly and there are no extra keys that are not supported.
- Check if class is in correct namespaces according to manifest and class name matches.
- Contract fields should be valid YAQL expressions.
- Methods blocks should be checked for MuranoPL code structure correctness. Method blocks may consist only of Expressions, Assignments and Block constructs. See full terms description for details.
- (Optional) Detect code that cannot be parsed as YAQL expression but is considered to be a valid expression.
UI validation¶
- UI definition should contain Application and Forms sections
- YAQL expressions in UI definition should be valid
- Form section should contain only valid forms http://docs.openstack.org/developer/murano/draft/appdev-guide/murano_packages.html#forms
Errors¶
Error code consists of prefix and error number. Prefix is an upper case alphabetical sequence. It can be empty for general errors. Errors specific for some validator should use validator short name as prefix. Error number is an upper case alphanumerical sequence (one single letter E or W following by three digits). Examples: E007, MPL:E042 for errors and W777, UI:W123 for warnings. Validation tool should have ability to produce documentation of supported errors and warnings.
Package structure reference¶
http://docs.openstack.org/developer/murano/draft/appdev-guide/murano_packages.html#package-structure
Package consists of:
- Manifest - File in package root directory named manifest.yaml.
- MuranoPL classes - YAML files located in Classes directory or its subdirectories (optional).
- UI definition - YAML file in UI directory (optional).
- Resources - files in Resources directory (optional).
- Logo (optional)
- images.lst file (optional)
Manifest structure¶
Format
Application package format. Can be specified as <FORMAT>/<VERSION>, where format is a format name string (can be MuranoPL or Heat.HOT). If format is omitted, MuranoPL is used by default.
- Required: no
- Default: MuranoPL/1.0
- Format: <FORMAT>/<VERSION>
Type
- Required: yes
- Type: Enum(Application, Library)
Name
Package human readable name
- Required: no
- Type: String
FullName
Package fully qualified name. Should be in lowerCamelCase notation separated by dots.
- Required: yes
- Type: String
Version
Version of package
- Required: no
- Default: 0.0.0
- Type: String, should have valid SemVer format
Description
- Required: no
- Type: String
Author
- Required: no
- Type: String
Tags
- Required: no
- Type: List(String)
Classes
- Required: no
- Type: Dict
- Keys: String, Fully qualified class name
- Values: String, Filename relative to Classes directory
Require
Package external requirements.
Required: no
Type: Dict
Keys: Dependent package FullName
- Values: null or string with exact version (‘1’, ‘2.3’, ‘4.5.6’) or
valid spec_strings according to python-semanticversion docs
UI
File with package UI definition
- Required: yes (for package type Application)
- Type: String, Filename relative to root directory
- Default: logo.png
Logo
Package logo in png format.
- Required: no
- Type: String, Filename relative to UI directory
- Default: ui.yaml
Meta
- Required: no
- Type: Dict or List(Dict) with metadata information
Class structure¶
Name
- Required: yes, if more than one class defined in the file
- Type: String
- Format:
- Matches Python class name format, except leading double underscore (__ClassName).
- Matches CamelCase naming style.
Properties
- Required: no
- Type: Dict
For more details see Class properties.
Methods
- Required: no
- Type: Dict
For more details see Class methods.
Extends
Single or multiple class names.
- Required: no
- Type: String | YAML List
Namespaces
- Required: no
- Type: Dict
- Keys: String; Namespace alias
- Values: String; Namespace full qualified name
- Format:
- Namespace alias is a valid YAQL identifier or =.
- Namespace FQN is a string divided by dots (.).
- Recommendation is to use lowerCamelCase naming style.
Import
- Required: no
- Type: String, Class name, List(Class names)
Usage
- Required: no
- Type: Enum(Class, Meta)
- Default: Class
Meta
- Required: no
- Type: Dict or List(Dict) with metadata information
MetaClass structure¶
In addition to regular Class structure, MetaClass (Class with Usage: Meta) can have additional properties:
Cardinality
- Required: no
- Type: Enum(One, Many)
Applies
- Required: no
- Type: Enum(Package, Type, Method, Property, Argument, All) or List(Enum)
Inherited
- Required: no
- Type: boolean
Class properties¶
Contract
- Required: yes
- Type: YAQL Expression
Usage
- Required: no
- Type: Enum
Usage | |
---|---|
In | Default |
Out | |
InOut | |
Const | |
Runtime | |
Config | Since version 1.1 |
Static | Since version 1.2 |
Default
- Required: no
- Type: Any value
Meta
- Required: no
- Type: Dict or List(Dict) with metadata information
Class methods¶
Scope
New in version 1.4.
- Required: no
- Type: Enum
- Values: Public, Session
Arguments
- Required: no
- Type: List(Dict)
For more details see Method’s Argument.
Body
- Required: no
- Type: YAQL Expression | List(YAQL Expression)
Usage
- Required: no
- Type: Enum
Usage | |
---|---|
Runtime | Default |
Static | Since 1.3 |
Extension | Since 1.3 |
Action | Deprecated since 1.4 |
Meta
- Required: no
- Type: Dict or List(Dict) with metadata information
Method’s Argument¶
- Type: Dict(String, Dict)
- Key: Argument name
Values:
- Contract: YAQL Expression
- Default: YAQL Expression
- Usage: Enum added since 1.4
Usage | |
---|---|
Standard | Default |
VarArgs | |
KwArgs |
Meta
- Required: no
- Type: List with metadata information
Extensions¶
Validation tool should be extensible and provide unified mechanism for builtin checks and extensions.
Extensions should be implemented as a python package and registered using entry points.
Extensions are loaded using stevedore library.
CLI Usage¶
Usage: mpl-check [options] <PACKAGE | DIRECTORY>
Arguments:
DIRECTORY Application working directory (e.g. contains
manifest.yaml, Classes/, etc.)
PACKAGE Compressed application (ZIP-archive)
Options:
--ignore=errors Skip errors and warnings.
--select=errors Check only for selected errors and warnings.
--scope=validators List of validators to execute
--strict Consider warnings as an errors.
Alternatives¶
Implement validation as part of Murano and validate the package when it is uploaded to Murano.
Pros:
- Easier to implement.
Cons:
- Suitable for Murano itself, but not for developers.
- Integration with CI which requires execution of functional tests and is slower than static validation.
- Cannot be integrated with text editors or IDEs.
Data model impact¶
None
REST API impact¶
Murano API for upload packages should use this tool to do package validation before store it. If package is not valid, HTTP error with code 400 should be returned to user with list of check errors in description.
Versioning impact¶
None
Other end user impact¶
- Murano application package developer will be able to validate package during development process.
Deployer impact¶
None
Developer impact¶
Application developer will be able to validate application code during development process without importing package to Murano and running deployment process or test suite.
Murano-dashboard / Horizon impact¶
Error message is displayed in Horizon when trying to upload malformed package.
Implementation¶
Application class¶
Application class provides an entry point to the checker. It is responsible for configuration, extensions loading and validation execution.
Application options are registered using oslo.config
Methods:
- __init__(self, options)
- load_plugins(self) - Finds validators from installed plugins
- prepare_check_suit(self, loaded_package, validator_list) - Collect prepared checks from all validators in validator_list against specified package. Returns list with prepared checks.
- list_errors(self, filters)- Lists errors/warnings exposed by all available validators
Validator classes¶
Validator class represents set for specific checks and logic to register them.
Classes
BaseValidator
+-- PackageValidator
+-- ManifestValidator
+-- UiValidator
+-- ClassValidator
Attributes
- short_name short name for validator. Used as prefix for validator specific errors
Methods
- __init__(loaded_package) - Init validator for specific format
- run(self) - Runs validator against specified suit of files or whole package if file_suit=None (for Package validator)
Checkers¶
Checker is a function that performs checking logic. It takes data from args and yields error if it is not valid. Checkers should not contain logic for data preparation. Checker can be reused multiple times with different options in a single suit of checks. Checker can be wrapped with custom filters (i.e. manifest version), so it is called only for matching conditions.
Exceptions¶
Class CheckerError contains information including error code, message, file name and position in that file. Additionally it can include code snippet, if it’s available. For YAML data it’s available using yaml.error.Mark class. Otherwise code snippet can be generated during error formatting.
Attributes
- code - Error code (Usually single letter and three digits)
- message - Human readable error reason
- filename - File name relative to package root
- line - Line number (starts from 0, optional)
- column - Column number (starts from 0, optional)
- source - One line code snippet with highlighted column (optional)
Package loaders¶
Package loaders provide interface to Murano packages in multiple formats. Currently packages can be loaded from directory, single file or zip archive package.
PackageLoader
+-- DirectoryLoader
+-- FileLoader
+-- ZipLoader
Methods
- try_load(cls, path, *options) - [classmethod] Tries to load package, if it can be loaded returns instance of loaded package, otherwise None.
Loaded package¶
Loaded package class introduces a general interface for access to packages of different format.
Methods
- get_file(self, name) - File wrapper if file exists, None otherwise
- format(self) - Ru
- exists(self, name) - Returns true if file exists in the package.
- list(self, subdir=None) - Returns list of all files in the package (or in it’s subdirectory).
- search_for(self, regex) - Return iterator of files fit to regex
File wrapper allows to access to raw file context with method raw() and to parsed dict with method parsed()
Formatters¶
Formatter classes help to represent error report in various ways. Errors can be printed to stdout/stderr using text representation, which suits developer or written to file in JSON/YAML format.
Assignee(s)¶
- Primary assignee:
- Alexander Saprykin <cutwater>
- Other contributors:
- Krzysztof Szukielojc <kszukielojc> Sergey Slipushenko <sslypushenko>
Work Items¶
- Develop core framework with support of extensions.
- Implement manifest and package structure checks.
- Implement Murano PL classes checks.
- Implement UI definition file validation.
- Implement YAQL checks.
- Integrate package validation with murano API.
- Develop CLI tool based on core framework
- Create documentation.
Testing¶
Package should include unit and functional tests.
Documentation Impact¶
Documentation for the library should include:
- Usage information that describes how to use the application.
- API reference.
- Plugin developer documentation.
References¶
None