QiM - Quality in Model-Driven Engineering

Project publications

"Evaluating Quality in Model-Driven Engineering"

Authors	Mohagheghi, P. and Aagedal, J.
Published	International Workshop on Modeling in Software Engineering MISE'07, Minneapolis USA, 19-20 May 2007. Published in Proceedings of ICSE 2007, ISSN 0270-5257.
Abstract	In Model-Driven Engineering (MDE), models are the prime artifacts, and developing high-quality systems depends on developing high-quality models and performing transformations that preserve quality or even improve it. This paper presents quality goals in MDE and states that the quality of models is affected by the quality of modeling languages, tools, modeling processes, the knowledge and experience of modelers, and the quality assurance techniques applied. The paper further presents related work on these factors and identifies pertinent research challenges. Some quality goals such as well-formedness and precision are especially important in MDE. Research on quality in MDE can promote adoption of MDE for complex system engineering.
Download	The paper is available here

"An Overview of Quality Frameworks in Model-Driven Engineering and Observations on Transformation Quality"

Authors	Mohagheghi, P. and Dehlen, V.
Published	2nd Workshop on Quality in Modeling, Nashville USA, 2 October 2007.
Abstract	Quality is often defined as fitness for purpose which is the key property to determine when evaluating quality. This paper presents some general requirements for evaluating quality frameworks. It also discusses characteristics of MDE that are important when building a quality framework, such as its use of models in several stages of development and maintenance, and its multi-abstraction level approach that requires consistency and traceability. MDE should strive for engineering of quality into software. As a step in extending quality frameworks for this purpose, we discuss integrating quality evaluation with quality engineering using MDE approaches. Applying it on transformations, we discuss both the transformability of models and the quality of transformations themselves. While completeness and precision of models are quality criteria for transformability of them, transformations should monitor and preserve model quality, be reusable, simple and efficient. We further discuss quality means and evaluation methods and propose future work.
Download	The paper is available here

"Where is the Proof? -A Review of Experiences from Applying MDE in Industry"

Authors	Mohagheghi, P. and Dehlen, V.
Published	Fourth European Conference on Model Driven Architecture ECMDA'08, Berlin Germany, 9-12 June 2008. Published in Lecture Notes in Computer Science LNCS 5095, ISSN 0302-9743.
Abstract	Model-Driven Engineering (MDE) has been promoted as a solution to handle the complexity of software development by raising the abstraction level and automating labor-intensive and error-prone tasks. However, few efforts have been made at collecting evidence to evaluate its benefits and limitations, which is the subject of this review. We searched several publication channels in the period 2000 to June 2007 for empirical studies on applying MDE in industry, which produced 25 papers for the review. Our findings include industry motivations for investigating MDE and the different domains it has been applied to. In most cases the maturity of third-party tool environments is still perceived as unsatisfactory for large-scale industrial adoption. We found reports of improvements in software quality and of both productivity gains and losses, but these reports were mainly from small-scale studies. There are a few reports on advantages of applying MDE in larger projects, however, more empirical studies and detailed data are needed to strengthen the evidence. We conclude that there is too little evidence to allow generalization of the results at this stage.
Download	The paper is available here

"A Metamodel for Specifying Quality Models in Model-Driven Engineering"

Authors	Mohagheghi, P. and Dehlen, V.
Published	Nordic Workshop on Model Driven Engineering NW-MoDE '08, Reykjavik Iceland, 20-22 August 2008.
Abstract	In the context of model-driven engineering, models act as the primary artifacts and are eventually transformed into other artifacts, such as code and documentation. Consequently, to evaluate and assure the quality of software systems, developers need tools and techniques that allow them to reflect upon the quality of the models themselves. This research paper discusses existing quality models before proposing a metamodel for specifying quality models in the context of model-driven engineering. A tool is being developed in Eclipse based on the proposed metamodel. For each project, developers can use the tool, consisting of predefined concepts relevant to quality, to select quality goals based on the context of their particular project. We will use the quality models to integrate previous work on the quality issues in model-driven engineering by relating the identified quality goals to quality-carrying properties, practices or means to achieve them, metrics for evaluation and collected empirical evidence.
Download	The paper is available here

"Towards a Tool-Supported Quality Model for Model-Driven Engineering"

Authors	Mohagheghi, P., Dehlen, V. and Neple, T.
Published	3rd Workshop on Quality in Modeling, Co-located with MODELS 2008, Toulouse, France.
Abstract	This paper reviews definitions of model quality before introducing five properties of models that are important for building high-quality models. These are identified to be correctness, completeness, consistency, comprehensibility and confinement. We have earlier defined a quality model that separates intangible quality goals from tangible quality-carrying properties and practices that should be in place to support these properties. A part of that work was to define a metamodel for developing quality models with MDE in mind. In this paper we analyze existing literature in order to extract model quality properties and to build a quality model with focus on the quality of models. For this purpose the metamodel is implemented in a tool that allows us to model quality models. The advantage of defining the metamodel is learning how to precisely define quality elements and relations in the quality model, and building models that may be used to generate documentation, guidelines or checklists. The disadvantage is mainly in the research phase where the metamodel is not stable and undergoes changes.
Download	The paper is available here

"Existing Model Metrics and Relations to Model Quality"

Authors	Mohagheghi, P. and Dehlen, V.
Published	ICSE Workshop on Software Quality 2009, WOSQ '09, 16 May, 2009.
Abstract	This paper presents quality goals for models and provides a state-of-the-art analysis regarding model metrics. While model-based software development often requires assessing the quality of models at different abstraction and precision levels and developed for multiple purposes, existing work on model metrics do not reflect this need. Model size metrics are descriptive and may be used for comparing models but their relation to model quality is not welldefined. Code metrics are proposed to be applied on models for evaluating design quality while metrics related to other quality goals are few. Models often consist of a significant amount of elements, which allows a large amount of metrics to be defined on them. However, identifying useful model metrics, linking them to model quality goals, providing some baseline for interpretation of data, and combining metrics with other evaluation models such as inspections requires more theoretical and empirical work.
Download	The paper is available here

"Definitions and Approaches to Model Quality in Model-Based Software Development – A Review of Literature "

Authors	Mohagheghi, P., Dehlen, V. and Neple, T.
Published	Journal of Information and Software Technology - Quality of UML Models, Volume 51, Issue 12, December 2009, Pages 1646-1669.
Abstract	More attention is paid to the quality of models along with the growing importance of modelling in software development. We performed a systematic review of studies discussing model quality published since 2000 to identify what model quality means and how it can be improved. From forty studies covered in the review, six model quality goals were identified; i.e., correctness, completeness, consistency, comprehensibility, confinement and changeability. We further present six practices proposed for developing high-quality models together with examples of empirical evidence. The contributions of the article are identifying and classifying definitions of model quality and identifying gaps for future research.
Download	The paper is available here

"Experiences of Developing a Network Modeling Tool Using the Eclipse Environment"

Authors	Andy Evans, Miguel A. Fernández, Parastoo Mohagheghi
Published	Model Driven Architecture - Foundations and Applications 2009, ECMDA-FA'09, Berlin.
Abstract	Domain-specific modeling solutions have been promoted for some time in order to improve the productivity of software developers by providing them with modeling environments that are easier to learn, integrate best solutions and provide the possibility to automate software development by generating code from models. This paper presents experiences of developing a network modeling tool in Telefónica using Eclipse GMF. A metamodel based on Common Information Model was used in this development. While we experienced benefits in terms of better usability by domain experts, we also faced challenges such as the high level of expertise required to develop a good enough language and tool, the shortcomings of the tools in providing support for modeling at different abstraction levels, and the difficulties in updating the modeling tool with changes in the metamodel. These challenges must be overcome before the tool can be a part of our development environment.
Download	The paper is available here