Simulated Execution Mode For Murano Engine¶
As an Application Developer I’d like to execute my workflows without actual deployment and interaction with murano-dashboard in order to verify my workflow before actual deployment by those increasing my application development speed.
Verifying application packages should be simple and fast. User doesn’t have to re-upload package and add app to the environment on every change.
- Allow application author to validate his application using unit-tests
- Those tests will be put to the application package to allow anyone to test this app at any time
- Tests will look like regular unit tests. Testing framework, which will run unit-tests will support commands, that will allow to load test package from directory, to call class methods and configure deployments parameters. That will make deployment test run easier. Also, test writer may run deployment several times, examine and compare results with different parameters
- Tests should be able to produce complete object model with some parameters
- environment attributes (such as tokens) (or overwriting values, defined in config);
- mock the methods of system classes which include various kinds of external communications. Dependent applications, system resources and various API clients and also be mocked. It should be allowed to specify a returned value. There would be separate specification for mocking, where the details will be described.
Test-case prototype may look like that:
Namespaces: =: io.murano.apps.foo.tests sys: io.murano.system pckg: io.murano.apps.foo Extends: io.murano.tests.TestFixture Name: FooTest Methods: initialize: Body: # - $.appJson: new(sys:Resources).json('foo-test-object-model.json') - $.appJson: - ?: id: 123 type: io.murano.apps.foo.FooApp name: my-foo-obj instance: ?: type: io.murano.resources.Instance id: 42 ... setUp: Body: - $.env: $.createEnvironment($.appJson) # creates an instance of std:Environment - $.myApp: $.env.applications.where($.name='my-foo-obj').first() - mock($.myApp.instance, "deploy", "mockInstanceDeploy", $this) - mock(res:Instance, deploy, "mockInstanceDeploy", $this) testFooApp: Body: - $.env.deploy() - $.assertEqual(true, $.myApp.getAttr('deployed')) tearDown: Body: mockInstanceDeploy: Arguments: - mockContext Body: - Return: # heat template
Provide one CLI command, that will mock creation of VMs and other things and returns the deployment result.
Cons: Impossible to verify deployments, where execution plan returns a value, which is used in future app workflow. Compare results of several deployments would be inconvenient Real VM’s can’t be
Data model impact¶
REST API impact¶
Tests will be placed to a package, so manifest version need to be updated. This functionality should be described in a separate spec. For now, there will be no impact on project itself.
Other end user impact¶
Murano-dashboard / Horizon impact¶
#. Add ‘simulation’ mode (new entry-point) to Murano Engine, where packages would be uploaded from the provided path there would be no interconnection with RabbitMQ
- Make changes to the class-loader, located in engine, to not use API. Separate spec is provided for this change (https://review.openstack.org/#/c/198745/).
- Implement testing framework, written in MuranoPL that will include the classes, described below. The structure would be taken from python unittest module. Framework will include test-runner
- Implement mock support.
- Define what is needed to change in MuranoPL itself
Testing framework may contain the following classes and methods. This are base classes for simple testing framework.
- TestCase class
|setUp()||Method called immediately before calling the test method.|
|tearDown()||Method called immediately after the test method has been called and the result recorded.|
|run(result=None)||Run the test, collecting the result into the test result object passed as result.|
|assert...||Different asserts (assertEqual, assertNotEqual, assertTrue, assertFalse).|
- TestResult class: This class is used to compile information about which tests have succeeded and which have failed.
|errors||A list containing 2-tuples of TestCase instances and strings holding formatted tracebacks. Each tuple represents a test which raised an unexpected exception.|
|failures||A list containing 2-tuples of TestCase instances and strings holding formatted tracebacks. Each tuple represents a test where a failure was explicitly signalled using the TestCase.assert*() methods.|
|testsRun||The total number of tests run so far.|
- TestRunner(stream=sys.stderr, descriptions=True, verbosity=1) A basic test runner implementation which prints results on standard error. Has run method, witch executes the given test case. Also stores the execution result.
For the fist time test may be run only one by one. Later we can add TestSuite class and TestLoader class: * TestLoader class is responsible for loading tests according to various criteria and returning them wrapped in a TestSuite (or TestSuite if will add this class).
|loadTestsFromTestCase(testCaseClass)||Return a suite of all tests cases contained in the TestCase-derived testCaseClass.|
- Implement simple mocking machinery
All mockes are separated into NonCallable and Callable mocks
|start()||Activate a patch, returning any created mock.|
|stop()||Stop an active patch.|
|patch(target)||The target is patched with a new object. target should be a string in the form package.module.ClassName.|
|attach_mock(mock, attribute)||Attach a mock as an attribute of this one, replacing its name and parent|
|configure_mock(kwargs)||Set attributes on the mock through keyword arguments|
initialize, __call__, _patch, __enter__, __exit__
New testing framework will be documented from scratch.
Discussions in IRC will be provided