Creating a cascading resource import structure for Robot Framework: Pt. 2/3 – The problem

This is the second post in a three-part series. The series will present a possible approach to overcome certain problems that are inherent to the mechanism, which the Robot Framework implements to facilitate the sharing of resources.

Part one described the various types of resources that can be reused in your Robot Framework (RF) projects and the mechanism that enables this sharing. See here for part 1. This second post will assume that you have read the first post and, therefore, are familiar with the specific terminology surrounding RF resources.

In this second post, we will take a closer look at the nature as well as the extent (or impact) of the problems that are surrounding the reuse of RF artifacts. The third (and final) post will then propose a simple, but effective and efficient, solution to these issues.

Technical heterogeneity requires numerous test libraries

In a heterogeneous, complex technical environment many test libraries must typically be imported, since multiple interfaces need to be addressed when implementing test automation against the various front-end and back-end components. For instance, at the time of writing, I am working within a single environment where I need to automate against SOAP, REST/HTTP, DB2, JMS as well as a web-GUI.

Lots of libraries are also required when automating the same components at multiple levels (gui, service and/or api) of the testing pyramid for different testing types, such as regression testing, system/integration testing, acceptance testing, end-to-end testing. For instance, at my current assignment, I am automating certain web-based components at the gui-level as well as at the service-level through REST and SOAP, in order to deliver test functions for the sake of various types of automated test designs.

In addition to the relevant interface driver libraries, a test project typically requires multiple convenience libraries (for the difference between interface driver libraries and convenience libraries, please see part 1 of this series). For any given test project, I will generally employ the following standard libraries: Builtin, Collections, OperatingSystem, String, DateTime and Process. Oftentimes also additional ones.

And even more such libraries are required when having to handle different in- and output formats, such as XML, JSON or others.

Functional richness and/or complexity requires numerous user defined resources

A product with even just a medium level of functional richness and/or complexity will require a lot of user keyword libraries and other types of user defined resources. Generally, several abstraction layers of test functions, wrappers, object maps (repositories), global variables and other such domain-specific resources will have to be implemented.

Depending on the scope of your project, in terms of the test types to automate, the product components to automate, etc., you will typically need dozens if not hundreds of functions. Consequently, depending on the levels of modularization you apply to your solution’s design (usually mirroring the functional/logical breakdown of your SUT), you may end up with dozens or sometimes even hundreds of user keyword libraries and other user defined resources.

That entails lots of importing

Please refer to part 1 of this series, in case you do not grasp any of the terms and concepts (such as test libraries, importing, user defined resources, etc.) that I am using in this section and that I assume to be familiar.

Generally, importing needs to be done within/at several layers/levels of your solution. Let’s have a look.

Importing test libraries within resource files

At the very beginning of your test project, you start out with just the resources that are available to you through the standard test libraries (such as the XML library) and/or through any external test libraries (such as the SUDSLibrary) that you installed. Generally you will then proceed to create your own, user defined resources. Typically, you will create these by implementing resource files. For instance resource files containing user keywords, i.e. files that contain your own, domain-specific test functions.

In this stage of your project, the latter can only be accomplished by reusing the existing resources in the available (standard and/or external) test libraries. However, this requires each of the test libraries that you need to reuse in a resource file, to be imported within that file. If you needed to create, for instance, a user keyword library containing (amongst others) functions that would have to call a SOAP service, get the input XML from a file and validate the output XML, you would have to import at least three libraries within that resource file: SUDSLibrary, XML library and OperatingSystem library. And that is merely on account of the one function: if the keyword library were to contain functions that required one or more other test libraries (e.g. to make http calls), you would then have to insert additional import statements to the keyword library file on account of those functions. Moreover, in almost all cases of creating user keyword libraries, you will employ several of the fundamental RF convenience libraries, such as the String library and the Collections library.

Accordingly, it is not uncommon to have between 5 and 10 import statements (all pertaining to test library imports) in a single resource file. Again, this depends on the complexity and diversity of your technical landscape. The more resource files you create, the more of these import statements you will end up with.

Figure 1 – Example of test library imports in a user keyword library.

Note that this bogus example is entirely unrealistic in that such a great amount of import statements is highly unusual (if not absurd). Moreover, the containing test project and everything in it, is (evidently) not representative of how a test project should be named, set up, organized and structured. 


Importing resource files within resource files

Identical to reusing a test library in a resource file, you will have to import any resource file that you want to reuse in the development of another/new resource file.

Typically, in the previous phase of test code development, you will have created a first, low-level layer of keyword libraries with technical-workflow-level functions and other such resources. For instance libraries implementing the so-called ‘glue-code’ (or ‘fixtures’) that connect to your domain code through some interface at the api, service and/or gui level. But you may also have created a set of convenience libraries (‘technical helper’ libraries) because your specific situation required convenience functions not provided by any of the various existing RF convenience libraries. Again, all of these user keyword libraries function at a lower, technical level. In addition to these user keyword libraries you will probably also have created other types of resources, such as variable files or object repositories.

After that first round of development, you will, in most cases, start to reuse the created resource files. Typically, you will create a layer of user keyword libraries (and other, related artifacts) that function at the workflow activity level and that will be re-using the keywords created at the technical-workflow level. Moreover, these workflow-activity level keyword libraries will probably, in turn, be re-used (and thus imported) by a thin layer of wrapper keywords (see, again, my other blog posts for a first introduction to test code layering).

Similar to what has been said in the previous section, with regard to reusing test libraries within your resource files, you can (and probably will) end up with multiple import statements throughout a lot of your workflow-activity-level resource files. All depending on the functional (and technical) richness and complexity of your domain on the one hand and the specific design of your test automation solution on the other hand.

Figure 2 – Example of user keyword library imports in a user keyword library.

Importing resource files within test suite files

When you are done building your test automation solution (i.e. the domain-specific part of your test automation framework), you will then start to create/add your automated test designs.

In case of the layering described above, you will use the test functions from the keyword libraries (and other domain-specific resources) that live at the workflow-activity level, to create your test cases/scenario’s. These keywords thus form the domain specific language in terms of which you can formulate your test scenario’s (please seethis post).

Test cases in RF are implemented as test suite files (see part 1). To be able to use your domain-specific keywords in a test case, the containing test suite file will have to import the relevant user defined resource file(s). The exact number of import statements in a test suite file is therefore dependent on the number of user keyword libraries that are required to specify all of the test cases that are to be contained in that test suite.

Generally, there may be between 5 and 10 import statements in a test suite file.

Please note: it is generally unnecessary to (directly) import test libraries into test suite files, since it is generally undesirable to insert technical low-level calls/statements into your test designs. However, there are situations in which this may become necessary. In those cases you will end up with even more import statements in your test suite files.

Figure 3 – Example of user keyword library imports in a test suite file.

And even: importing resource files within logical test suite folders

As mentioned before, if you are not familiar with the notion of a ‘logical test suite’/‘test suite folder’, please refer to the referenced blog posts.

A test suite folder may feature its own set of import statements. Specifically, it will require one or more of these statements, when the need arises for suite-level set-up and/or tear-down routines. And/or in the case of having to specify default test setup/teardown routines.

These setup/teardown routines will, generally, be domain-specific. Therefore, these routines will have been implemented through one or more user keywords, within one or more user keyword library files. Therefore, all of the relevant user keyword libraries need to be imported within a test suite folder.

Figure 4 – Example of user keyword library imports in a test suite folder.

Alas: No inheritance of ‘higher-level’ import statements.

Just as is the case with programming languages such as Java or Python and as is the case with other proprietary scripting languages (such as that of FitNesse), there is no ‘inheritance’ of import statements within RF test code.

Notwithstanding, many people, especially those without prior programming experience, are often surprised by this lack of inheritance. Moreover, they often (initially) expect and assume inheritance and therefore see their code fail due to missing import statements.

However, RF (similar to other platforms and frameworks) requires you to create an import statement for each resource file (e.g. a test library, a user defined keyword file, etc.) in each file (e.g. a test suite file, a user defined keyword file, etc.) that needs to reuse that resource file.

So even if, for instance, you create an import statement for the XML test library at the level of a test suite folder, then each child of that folder (such as a test suite sub-folder, a test suite file and/or a user keyword file), nevertheless needs to have its own import statement in case it needs to reuse the XML Library. The same goes for the importing of any other resource!

Figure 5 – No import ‘inheritance.

That’s why you might end up with hundreds of import statements. Or more!

Depending on the factors described above (i.e. the technical and functional complexity and size of your domain; scope of your automation project; your test strategy; etc.), you can end up with multiple/many import statements in each of a sizable set of files. Given an average of 6 import statements (which is not uncommon in my experience), when you’d have merely 30 user keyword libraries, you’d end up with 180 import statements. Many of which will be duplicates.

In my experience, when working in an enterprise-level project, there are often not only more than 6 import statements per file (on average), but also many, many more user keyword libraries than those 30. Thus, I regularly work on projects with many hundreds of import statements across the test code base. It would not be too much of a stretch of the imagination, to assume that there may be projects out there that even have thousands of these statements.

Result: maintaining your imports can be an effort in and of itself

Having to work with that many import statements will be confounding and thus lead to mistakes. It will be an effort in and of itself to maintain this set of import statements and keep it efficient, consistent and relevant.

For instance, if you wanted to change an import parameter, you would have to change it many times throughout the code base. If you wanted to change the name of a user defined resource file, you would have to change each statement that imports that resource. A real-life example would be a migration from the Selenium2Library to the SeleniumLibrary: the name change will lead to maintenance in maybe hundreds of files. Of course you can find and replace or even create a script to do the work for you. But that may always lead to errors/omissions and still means work to do.

In the next, and last, post, I will propose a very simple, yet effective mechanism to reduce that work to an absolute minimum.

Naar het overzicht

Alain Bultink | Directeur

Carlo van Driel | Directeur

Theo Janssen | Business Development