For MDSD there are a lot of modeling options and modeling tools available. You can do

• textual modeling with a stand-alone modeling language (aka external DSL)
• graphical modeling with UML
• graphical modeling without UML
• modeling with a general purpose programming language (aka internal DSL)
• tabular modeling (kind of a spread sheet)
• tree structured modeling (XML for instance)

For the generation part there are also a lot of tools available to do the job. Most of them apply a template approach where you express generation logic with the help of a template language.

Looking at how many years the model-driven approach for software development exists – CASE-tools in the 1990ies and MDA/MDSD since 2000 – and knowing how many different tools there were available on the market – and are not anymore – one may say: “MDSD has turned out not to fulfill its promises”. I would rather say: “It has been brought back down to earth”.

Let me try to outline some critical aspects of model-driven software development. By doing so you may get an idea of how MDSD, in many cases, can easily work when you do the following:

• use an API-based, component-oriented, open source code generation tool that is integrated well into your IDE – JenerateIT is one example
• for your modeling tasks and for the development of generation logic, use the general purpose programming language that you anyway use for your software under development
• re-use and extend existing meta-models and generation logic that is provided by the community

Costs for MDSD Tools

There were and are modeling and generation tools out there that cost a lot of money. While a few of those tools may be worth the money, there are also lots of other tools that are open source and are of value. Ideally, there is a tool that comes with no licensing costs and at the same time doesn’t require a lot of effort to learn and comprehend how it works and how it should be used and adapted.

It is easy to find out whether you can use a tool without licensing costs. It is much more difficult, though, to find out, which additional, hidden costs are involved in using such a tool.

As a matter of fact, if you use Java (or another object-oriented programming language) as your meta-meta-model and use your IDE as modeling tool, you do not have to pay (additional) money for a modeling tool. Also, you do not have to spend additional efforts for installation, set-up and maintenance of a modeling tool.
Doing so does not limit you to use your IDE as modeling tool. You are still free to add another modeling tool to your MDSD tool-landscape. In that scenario, a model-to-model transformation converts the model (created with the new tool) to a model that conforms to the meta-model that is understood by your existing generation logic.

Quick and smooth Introduction of MDSD

When you start applying MDSD, you are confronted with the following tasks.

• install and configure modeling and generation tool
• learn how to use modeling tool
• learn how to use generation tool
• learn how to design and implement modeling language
• design of modeling language
• implement modeling language
• learn how to develop generation logic
• develop generation logic
• add generation steps to your continuous integration infrastructure
• create and maintain a reference model for the development and testing of your generation logic
• develop unit tests for generation logic

You do not only have to handle those tasks once. There also need to be people who are responsible for some of those tasks, not only at the begining of a project but constantly. The sheer number of tasks together with the uncertainty on what risks and efforts are involved with them, acts as a deterrent.

By using your IDE as modeling tool and using a general purpose programming language as the language to develop generation logic, you get rid of some of those points and risks and make some of them being not deterrent.

Flexibility and Agility in Modeling and Generation

This is a double-edged sword. You may find it helpful, if the modeling or generation tool limits the options to design a modeling language or to develop, maintain and control generation logic. On the other hand, you loose agility and flexibility due to those limitations. The more often you ask “Can I do this or that with tool x?” and you get answers like “No” or “Yes, but you have to use a hack/trick” or “Yes, but it is slow” or “No, but in a year this may be possible”, you should consider using a different tool or a different approach.

With regards to flexibility in code generation, the following features may be worth thinking about.

• non-sequential writing to an artifact, e.g. writing a Java import statement while writing a method body
• from inside generation logic, having the ability to find/access objects that represent other artifacts that are being generated and objects that encapsulate other generation logic
• back-tracking to easily find out and visualize, which part of generation logic is responsible for the writing of which part of an artifact
• full access to the content of the previously generated artifact, including potential manual changes therein

JenerateIT is capable of doing all this … and more.

Reuse and Encapsulation of Generation Logic

Often it is difficult – if not impossible – to reuse existing generation logic. This is escpecially true when template-based generation is applied. When generation logic is implemented by means of an object-oriented general purpose programming language like for instance Java, support for reuse is built-in.

Reuse does not only mean that you may use existing generation logic for more than one project or more than one piece of software. It also means, that you do not need to repeat yourself when developing generation logic. Lets assume we generate several artifacts. How does the generation logic for artifact A know what is going to be written to artifact B? A basic solution may simply know (by convention), what is getting written to artifact B. A more sophisticated solution, in a generation tool that uses not templates but an API to write generation logic, generation logic for artifact A would find out, what is getting written to B.

With the API approach, tasks like the following are getting easy to implement:

• find fully-qualified names of all Java classes that are going to be generated
• find all internationalization files that are going to be generated
• find the fully-qualified name of the class that holds data for data binding for a graphical user interface
• find names of all files that are going to contain the online help

JenerateIT‘s API is capable of doing all this. The generation logic of gApp makes extensive use of those capabilities.

Flexible Models

In MDSD a model is an instance of a meta-model. And a meta-model is described by means of a meta-meta-model. An EMF meta-model for instance consists of objects of type EClass, EAttribute, EPackage, etc. (all Ecore types). A meta-model created with MetaEdit+ consists of objects of type Graph, Object, Relationship, Role, Port and Property (GOPPRR).

That’s all fine. However, what if you want to use (existing) data/information that has not been created by means of an MDSD modeling tool? You have to spend extra efforts to somehow transform that data/information to be available in a form that is based on Ecore, GOPPRR or whatever meta-meta-model you use for modeling.

Things become different when you use the Java language’s model as the meta-meta-model for your meta-models. With this, everything that can be read by means of Java is a model. Defining a meta-model then is equivalent with writing Java classes. Creating a model based on that meta-model is equivalent with creating instances of those Java classes. Your Java-IDE automatically turns into a modeling tool. All Java support in your IDE also supports your modeling tasks. No apprehension that your modeling tool will not be available on the market anymore or that it won’t be under further development.
In addition to this, your knowledge and experience about the Java language and libraries and object-orientation (OOA/OOD/OOP) instantly can be leveraged. JenerateIT fosters this way of modeling, while using models based on other formats like EMF, UML, XML, etc. is still possible and supported.

Mixing generated and hand-crafted Code

Most experts for MDSD say to generate at most 80% of your code is a realistic scenario. So you have at least 20% manually written code (note that “manually written” can also mean “automatically created through your IDE”). There are a few well-known solutions to mix generated with hand-written code.

• generation gap pattern
• protected regions/areas/code (aka developer areas)
• one-off targets – artifacts that will never be re-generated once their content has been manually modified (in fact that is a special case of protected regions/areas/code)

While it is good to have those solutions, additional, more sophisticated, specialized solutions are required to efficiently and comfortably mix generated with manually written code. And those solutions do not require protected regions/areas/code.

• protect manual changes in a property file from being overwritten (e.g. for application internationalization)
• merge manually added Java import statements with import statements that are being created by generation logic
• protect manual changes in originally empty method bodies
• protect manually added fields or methods of a Java class
• … further specific solutions …

Note that for those solutions to work, it is required that the content of previously generated artifacts are available during generation. And using a general purpose programming language to write generation logic makes it easy and straightforward to handle these things. JenerateIT does exactly that.

Integration in Development-Environment

Some MDSD tools are stand-alone tools. Using those tools makes it difficult to integrate them into a software developer’s IDE. It is still possible to work with the tools and the IDE. But the acceptance of software developers may be low. Also, due to the fact that a stand-alone tool runs in a separate process, synchronization tasks have to be executed. And such tasks may have a negative effect on the development round-trip and generation performance.
A better tooling solution is to have a modeling and generation environment, that is integrated in the IDE. And even better than this is a solution that provides the same modeling and generation support accross different IDEs for the same target platform (e.g. Eclipse, NetBeans and IntelliJ for the Java platform).

JenerateIT is well integrated into Eclipse. Integrations into NetBeans and IntelliJ are easy to accomplish since the core of JenerateIT is not dependent on any of those IDEs. This modularization also allows for running an Ant-based generation, for instance as part of the continuous integration process.

Teamwork and Model-Partitioning

One of the biggest challenges in the modeling part of MDSD is how to model in a team. Things like partitioning of models, model diffs, model versioning and model merge are not easy to accomplish. MDSD modeling tool providers typically have their proprietary solution to this problem. And that solution normally costs a lot of money.
When you use textual modeling, no matter whether it is through internal DSLs on top of a general purpose programming language or through an external DSL (e.g. by using Eclipse Xtext), teamwork works the same way as teamwork in traditional software development works. Any version control tool that is capable to do versioning, diffing and merging of textual artifacts can be used to store the model. Open source examples for such tools are Apache Subversion and Git.

Conclusion

Doing model-driven software development can be really easy, without the burden of complex theory and without a lot of new things to learn and digest. It is not necessary to completely bend your development environment and yourself. The biggest challenge that remains is to add at least two new levels to your thinking: meta-model and model, together with transformation/generation steps to get utilizable results. That’s the price you have to pay to gain a lot of power, flexibility and agility.

You may want to give the open source code generation tool JenerateIT a try, to experience the simplicity of MDSD.

Here is a link to a video on youtube that nicely summarizes the event.

A lot of attention was given to domain specific modelling, especially to the question of whether or when textual, graphical, tabular or form-based modelling is best suited for the modelling of software systems. The number of choices and quality of modelling tools seems to become bigger theese days. That’s good news since it makes it easier to find a modelling tool that is best suited to solve a given problem.

On the last conference day, I was speaking about Legacy Application Modernization – Experiences in a real-world MDA/MDSD Project. The people in the audience were asking quite some questions that I was happy to answer thoroughly. One listener had approached me after my talk and we had a fruitful discussion on modelling and code generation.

Last but not least, I took the chance to collect new ideas through the course of the conference to improve our own tools for model-driven software development. Those ideas are already part of our release planning process.

See you next year in Cambridge. Code Generation 2010 surely has something for you, too.

Martin Fowler is currently working on a book on DSLs. On the web you can find his Work In Progress on the book. The sheer quantity of information that you find there gives you an idea of this issue’s complexity.

External versus Internal DSL

Please note for the further reading, that there is a difference between an internal and an external DSL. On JAVAWORLD, Venkat Subramaniam gives a short and comprehensible explanation of the difference between an internal and an external DSL:

Internal DSLs ride on a host language, so an internal DSL’s syntax is both influenced and restricted by the host language. External DSLs can be built from the ground up but doing so requires mental muscle, as well as a very good parser.

As a developer you normally are confronted with situations where you have to use one or more external DSLs at the same time, besides your general purpose programming language. Even if you don’t do model-driven software development, this is true, for instance when you write SQL statements in your program. SQL is an external DSL for the domain “create, read, update and delete persistent data”.

In his article on Language Oriented Programming, Sergey Dmitriev (from JetBrains) compares class libraries with DSLs:

… You might say that a class library corresponds to some domain. Lo and behold, we see the connection. Class libraries are wannabe DSLs! …

Class libraries in the broadest sense represent internal DSLs.  Now, the question is: are we better off using external DSLs or using internal DSLs? In terms of SQL you could use the promising open source framework Squill to convert the external DSL SQL into an internal DSL in order to simplify coding and to make the code more readable.

In the aforementioned “Work in Progress” of Martin Fowler’s book, he elaborates on when it is better to use internal DSLs and when to use external DSLs.

Choose carefully

There are a lot of tools available to design and define your own external DSLs. Like for instance Meta Programming System, MetaEdit+, ANTLR, Xtext, to mention some well known of them.

With feature-rich DSL tools at hand, you could be swift to invent new external DSLs. Eventually, you will have to carefully choose the external DSLs to be used and possibly invented for your software development. The same holds true for the DSL tool to be used. The added complexity potentially could outweigh the gained expressiveness. Applied properly, using DSLs boosts productivity and moreover bridges the gap between software developers and those persons who provide requirements. Then DSLs don’t lead to a Babylonian Language Confusion but rather to the opposite: they help all people who are involved in the analysis, design and development of a software system to better understand each other.

- Monday, 26th of January -

Domain-Specific Modelling: Enabling full code generation
(Juha-Pekka Tolvanen, MetaCase)
In this track I will get a good feeling for how easy it is to design, implement and use domain specific languages with the help of MetaEdit+. Models created by means of MetaEdit+ could be used by JenarateIT to generate something useful from them.
At least as interesting for me, is to see, how the participants of that track manage to use the tool and the methodology. This way I can get a good idea on how easy or difficult it is, to use MetaEdit+ compared to similar tools.

- Tuesday, 27th of January -

User-Interface Programming with external DSLs
(Sven Efftinge, itemis)
(Jan Köhnlein, itemis)
The presenters show how an existing application, based on Oracle-Forms, had been modernized by porting it to a Java solution. For the mapping of database tables to forms, an external DSL had been used, together with a project-specific IDE.
I am curious to see, what the external DSL looks like and what the presenters will tell about how the developers managed to use the DSL and the model-driven development approach. Maybe I can learn something that can add to the GS Solution Platform and to the way the model-driven development approach can be propagated more easily.

Architecture Models: When to talks about them, and when not to
(Dan Cocks, Lockheed Martin Corporation)
I think in this track I will learn something on how to speak about modelling and models in situations where conversational partners are non-modellers and potentially have negative attitudes towards modelling.

- Wednesday, 28th of January -

The seven deadly sins of UML modelling
(Andreas Ditze, MID)
Getting to know more about problems that arise from doing UML modelling in an inefficient or unhelpful way will help me to convince people to rather use specialized DSLs or at least to prevent people from running into such problems.

- Thursday, 29th of January -

Architecture as Language – Modelling of Architecture by means of textual DSLs
(Markus Völter, itemis)
In combination with an architecture, a textual DSL is getting defined that supports that architecture. Models created with that DSL then can be automatically processed.
I presume openArchitectureWare and xText are presented here. To experience how xText is being used here will give me a better feeling for it. This way I can better assess, whether xText is a good choice for providing input models for JenerateIT.

Model-driven Software Development – a practical Experience Report
(Nicolas Cabanis, SYNLOGIC)
Reports on practical experiences gained in a real project are rare. All the more I have some expectations to learn something new about best practices. I am especially curious to see cause and effect in the context of the project environment that the presenter was working in.

In this weblog you will find entries written by people of Generative Software. An entry’s content represents the opinion of the person having contributed the entry.