Providing software maintenance and evolution as a service in a small organization: an approach based on CMMI-DEV and CMMI-SVC

This paper evaluates the adoption of CMMI-DEV and CMMI-SVC in small organization for the improvement of software maintenance and evolution process. A Software process improvement (SPI) initiative was performed in a Brazilian small sized software product maintenance organization. We used the Action-Research methodology to evaluate the viability, benefits and lessons learned from the simultaneous adoption of these models. As a result we observed that a set of Process Areas from CMMI-SVC were relevant for supporting the management software maintenance and evolution activities, while Process Areas from CMMI-DEV were relevant for supporting its technical aspects.

  • [PDF] R. Moreira, M. Souza, Y. C. Cavalcanti, A. C. Rouiller, and A. Vasconcelos, “Providing software maintenance and evolution as a service in a small organization: an approach based on cmmi-dev and cmmi-svc,” in XVIII Ibero-American Conference on Software Engineering (CIBSE’2015), 2015, pp. 750-763.
    [Bibtex]
    @INPROCEEDINGS{MoreiraCIBSE2015,
    title = {Providing software maintenance and evolution as a service in a small organization: an approach based on CMMI-DEV and CMMI-SVC},
    author = {Moreira, R. and Souza, M. and Cavalcanti, Y. C. and Rouiller, A. C. and Vasconcelos, A.},
    booktitle = {{XVIII Ibero-American Conference on Software Engineering (CIBSE'2015)}},
    abstract = {},
    year = {2015},
    pages = {750--763},
    }
Share

Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests

In 29th IEEE/ACM International Conference on Automated Software Engineering (ASE 2014), Vastëras, Sweden.

Abstract: Change Requests (CRs) are key elements to software maintenance and evolution. Finding the appropriate developer to a CR is crucial for obtaining the lowest, economically feasible, fixing time. Nevertheless, assigning CRs is a labor-intensive and time consuming task. In this paper, we present a semi-automated CR assignment approach which combines rule-based and information retrieval techniques. The approach emphasizes the use of contextual information, essential to effective assignments, and puts the development team in control of the assignment rules, toward making its adoption easier. Results of an empirical evaluation showed that the approach is up to 46,5% more accurate than approaches which rely solely on machine learning techniques.

  • [PDF] [DOI] Y. C. Cavalcanti, I. D. C. Machado, P. A. M. S. da Neto, E. S. de Almeida, and S. R. L. de Meira, “Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests,” in Proceedings of the 29th ieee/acm international conference on automated software engineering (ase’2014), 2014, pp. 608-615.
    [Bibtex]
    @INPROCEEDINGS{CavalcantiASE2014,
    title = {{Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests}},
    author = {Yguaratã Cerqueira Cavalcanti and Ivan Do Carmo Machado and Paulo Anselmo da Mota Silveira Neto and Eduardo Santana de Almeida and Silvio Romero de Lemos Meira},
    booktitle = {Proceedings of The 29th IEEE/ACM International Conference on Automated Software Engineering (ASE'2014)},
    abstract = {Change Requests (CRs) are key elements to software maintenance and evolution. Finding the appropriate developer to a CR is crucial for obtaining the lowest, economically feasible, fixing time. Nevertheless, assigning CRs is a labor-intensive and time consuming task. In this paper, we present a semi-automated CR assignment approach which combines rule-based and information retrieval techniques. The approach emphasizes the use of contextual information, essential to effective assignments, and puts the development team in control of the assignment rules, toward making its adoption easier. An empirical validation was performed and the results pointed out that the approach is up to 46,5% more accurate than approaches which rely solely on machine learning techniques.},
    year = {2014},
    month = {September},
    doi = {10.1145/2642937.2642964},
    pages = {608--615},
    }
Share

The bug report duplication problem: an exploratory study

Published in Software Quality Journal, see it.

Abstract. Duplicate bug report entries in bug trackers have a negative impact on software maintenance and evolution. This is due, among other factors, to the increased time spent on report analysis and validation, which in some cases takes over 20 min. Therefore, a considerable amount of time is lost in duplicate bug report analysis. In order to understand the possible factors that cause bug report duplication and its impact on software development, this paper presents an exploratory study in which bug tracking data from private and open source projects were analyzed. The results show, for example, that all projects we investigated had duplicate bug reports and a considerable amount of time was wasted by this duplication. Furthermore, features such as project lifetime, staff size, and the number of bug reports do not seem to be significant factors for duplication, while others, such as the submitters’ profile and the number of submitters, do seem to influence the bug report duplication.

  • [DOI] Y. C. Cavalcanti, P. A. M. S. da Neto, D. Lucrédio, T. Vale, E. S. de Almeida, and S. R. L. de Meira, “The bug report duplication problem: an exploratory study,” Software quality journal, vol. 21, iss. 1, pp. 39-66, 2013.
    [Bibtex]
    @ARTICLE{CavalcantiSQJ2013,
    Author = {Yguaratã Cerqueira Cavalcanti and Paulo Anselmo da Mota Silveira Neto and Daniel Lucrédio and Tassio Vale and Eduardo Santana de Almeida and Silvio Romero de Lemos Meira},
    Title = {The bug report duplication problem: an exploratory study},
    Journal = {Software Quality Journal},
    Publisher = {Springer Netherlands},
    Issn = {0963-9314},
    Keyword = {software maintenance, bug report duplication, bug tracking, change request},
    Pages = {39-66},
    Volume = {21},
    Number = {1},
    Year = {2013},
    Url = {http://dx.doi.org/10.1007/s11219-011-9164-5},
    doi = {10.1007/s11219-011-9164-5},
    Abstract = {Duplicate bug report entries in bug trackers have a negative impact on software maintenance and evolution. This is due, among other factors, to the increased time spent on report analysis and validation, which in some cases takes over 20 min. Therefore, a considerable amount of time is lost in duplicate bug report analysis. In order to understand the possible factors that cause bug report duplication and its impact on software development, this paper presents an exploratory study in which bug tracking data from private and open source projects were analyzed. The results show, for example, that all projects we investigated had duplicate bug reports and a considerable amount of time was wasted by this duplication. Furthermore, features such as project lifetime, staff size, and the number of bug reports do not seem to be significant factors for duplication, while others, such as the submitters’ profile and the number of submitters, do seem to influence the bug report duplication.}
    }
Share

On Strategies for Testing Software Product Lines: A Systematic Literature Review

Published in Information and Software Technology journal, see the early view.

Context. Testing plays an important role in the quality assurance process for software product line engineering. There are many opportunities for economies of scope and scale in the testing activities, but techniques that can take advantage of these opportunities are still needed.

Objective. The objective of this study is to identify testing strategies that have the potential to achieve these economies, and to provide a synthesis of available research on SPL testing strategies, to be applied towards reaching higher defect detection rates and reduced quality assurance effort.

Method. We performed a literature review of two hundred seventy-six studies published from the year 1998 up to the 1st semester of 2013. We used several filters to focus the review on the most relevant studies and we give detailed analyses of the core set of studies.

Results. The analysis of the reported strategies comprised two fundamental aspects for software product line testing: the selection of products for testing, and the actual test of products. Our findings indicate that the literature offers a large number of techniques to cope with such aspects. However, there is a lack of reports on realistic industrial experiences, which limits the inferences that can be drawn.

Conclusion. This study showed a number of leveraged strategies that can support both the selection of products, and the actual testing of products. Future research should also benefit from the problems and advantages identified in this study.

  • [DOI] I. C. do Machado, J. D. McGregor, Y. C. Cavalcanti, and E. S. de Almeida, “On Strategies for Testing Software Product Lines: A Systematic Literature Review,” Information and Software Technology (online first), vol. 56, iss. 10, pp. 1183-1899, 2014.
    [Bibtex]
    @ARTICLE{CarmoMachado2014,
    title = {{On Strategies for Testing Software Product Lines: A Systematic Literature Review}},
    journal = {{Information and Software Technology} (online first)},
    volume = {56},
    number = {10},
    pages = {1183--1899},
    year = {2014},
    issn = {0950-5849},
    doi = {10.1016/j.infsof.2014.04.002},
    url = {http://www.sciencedirect.com/science/article/pii/S0950584914000834},
    author = {Ivan do Carmo Machado and John D. McGregor and Yguaratã Cerqueira Cavalcanti and Eduardo Santana de Almeida},
    }
Share

Low-level variability support for web-based software product lines

The Web systems domain has faced an increasing number of devices, browsers, and platforms to cope with, driving software systems to be more flexible to accomodate them. Software product line (SPL) engineering can be used as a strategy to implement systems capable of handling such a diversity. To this end, automated tool support is almost indispensable. However, current tool support gives more emphasis to modeling variability in the problem domain, over the support of variability at the solution domain. There is a need for mapping the variability between both abstraction levels, so as to determine what implementation impact a certain variability has. In this paper, we propose the FeatureJS, a FeatureIDE extension aiming at Javascript and HTML support for SPL engineering. The tool combines feature-oriented programming and preprocessors, as a strategy to map variability at source code with the variability modeled at a higher level of abstraction. We carried out a preliminary evaluation with an industrial project, aiming to characterize the capability of the tool to handle SPL engineering in the Web systems domain.

  • [PDF] [DOI] I. C. do Machado, A. Santos, Y. C. Cavalcanti, E. Trzan, M. Souza, and E. de Almeida, “Low-level variability support for web-based software product lines,” in Proceedings of the 8th international workshop on variability modelling of software-intensive systems (vamos’2014), Nice, France, 2014.
    [Bibtex]
    @INPROCEEDINGS{MachadoVAMOS2014,
    author = {Ivan do Carmo Machado and Alcemir Santos and Yguaratã Cerqueira Cavalcanti and Eduardo Trzan and Márcio Souza and Eduardo  Santana de Almeida},
    title = {Low-level Variability Support for Web-based Software Product Lines},
    booktitle = {Proceedings of the 8th International Workshop on Variability Modelling of Software-intensive Systems (VAMOS'2014)},
    year = {2014},
    doi = {10.1145/2556624.2556637},
    keywords = {Software Product Lines, Software Variability, Software Traceability},
    address = {Nice, France},
    }
Share

Challenges and opportunities for software change request repositories: a systematic mapping study (JSEP journal)

Published in Journal of Software: Evolution and Process. See the early view.

Software maintenance starts as soon as the first artifacts are delivered and is essential for the success of the software. However, keeping maintenance activities and their related artifacts on track comes at a high cost. In this respect, change request (CR) repositories are fundamental in software maintenance. They facilitate the management of CRs and are also the central point to coordinate activities and communication among stakeholders. However, the benefits of CR repositories do not come without issues, and commonly occurring ones should be dealt with, such as the following: duplicate CRs, the large number of CRs to assign, or poorly described CRs. Such issues have led researchers to an increased interest in investigating CR repositories, by considering different aspects of software development and CR management. In this paper, we performed a systematic mapping study to characterize this research field. We analyzed 142 studies, which we classified in two ways. First, we classified the studies into different topics and grouped them into two dimensions: challenges and opportunities. Second, the challenge topics were classified in accordance with an existing taxonomy for information retrieval models. In addition, we investigated tools and services for CR management, to understand whether and how they addressed the topics identified.

  • [DOI] Y. C. Cavalcanti, P. A. da Mota Silveira Neto, I. C. do Machado, T. F. Vale, E. S. de Almeida, and S. R. de Lemos Meira, “Challenges and opportunities for software change request repositories: a systematic mapping study,” Journal of software: evolution and process, vol. 26, iss. 7, pp. 620-653, 2013.
    [Bibtex]
    @ARTICLE{CavalcantiJSEP2013,
    author = {Cavalcanti, Yguaratã Cerqueira and da Mota Silveira Neto, Paulo Anselmo and Machado, Ivan do Carmo and Vale, Tassio Ferreira and de Almeida, Eduardo Santana and de Lemos Meira, Silvio Romero},
    title = {Challenges and opportunities for software change request repositories: a systematic mapping study},
    journal = {Journal of Software: Evolution and Process},
    issn = {2047-7481},
    volume = {26},
    number = {7},
    url = {http://dx.doi.org/10.1002/smr.1639},
    doi = {10.1002/smr.1639},
    pages = {620--653},
    keywords = {software maintenance, software evolution, software quality assurance, change request repository, bug report, bug tracking},
    year = {2013},
    abstract = {Software maintenance starts as soon as the first artifacts are delivered and is essential for the success of the software. However, keeping maintenance activities and their related artifacts on track comes at a high cost. In this respect, change request (CR) repositories are fundamental in software maintenance. They facilitate the management of CRs and are also the central point to coordinate activities and communication among stakeholders. However, the benefits of CR repositories do not come without issues, and commonly occurring ones should be dealt with, such as the following: duplicate CRs, the large number of CRs to assign, or poorly described CRs. Such issues have led researchers to an increased interest in investigating CR repositories, by considering different aspects of software development and CR management. In this paper, we performed a systematic mapping study to characterize this research field. We analyzed 142 studies, which we classified in two ways. First, we classified the studies into different topics and grouped them into two dimensions: challenges and opportunities. Second, the challenge topics were classified in accordance with an existing taxonomy for information retrieval models. In addition, we investigated tools and services for CR management, to understand whether and how they addressed the topics identified. Copyright © 2013 John Wiley & Sons, Ltd.},
    }
Share

New Version of RiSE Latex Template

I have uploaded a new version of the RiSE Latex Template. It included capabilities for the listings package, which is very useful to display source code blocks in the text. The template automatically inserts the listings in the Table of Contents, as well as it provides a List of Listings after the List of Acronyms. You are free to change this order according to your needs. I would like to thank Kalil Bispo, who provided all the necessary latex code for this modification.

The migration for this new version is very simple. You just need to update the risethesis.cls file and include the command \lstlistoflistings, i.e. after the list of acronyms.

The RiSE Latex Template has been widely used for writing thesis in the Center for Informatics at UFPE. Thus, it is necessary to continuously improve the template. If you have suggestions, please fill a change request here.

Share

Towards Understanding Software Change Request Assignment: A survey with practitioners — EASE 2013

Context: Change Request (CR) repositories play an important role in the software maintenance and evolution process. Through a CR repository, software changes are reported and assigned to developers. Finding the appropriate developer to a CR is crucial for obtaining the lowest, economically feasible, fixing time. Nevertheless, assigning CRs is a labor-intensive and time consuming task. Although many work have proposed automated approaches for CR assignment, they have been implemented without investigating the fundamental aspects which characterize the task itself.

Objective: This paper investigates the impact of CR assignment on developers productivity and identifies the fundamental aspects that characterize it, such as the strategies to perform the assignments and the complexity involved in them. Such investigation improves the current knowledge on the topic, providing researchers and practitioners with useful information towards developing effective solutions.

Method: A survey was performed with software developers to understand CR assignment in the Brazilian Federal Organization for Data Processing. The questionnaire was composed of 38 questions, being them both open-ended and closed-ended. We analyzed the answers of 36 respondents.

Results: We find that: there is a significant amount of time being spent on assignments (e.g., assigning 20 CRs can take up to 3.3 hours); there are many strategies used to assign CRs, which are complementary to those used in current automated solutions; and CR assignment is very complexity due to a process that requires cognitive abilities for information seeking, communication, and memorization.

Conclusion: CR assignments are fundamental to software maintenance, however it is an expensive activity. Although we understand that fully and totally accurate automation is unlikely, further improvements on this direction are feasible and necessary to reduce costs. This way, this paper brings relevant findings to guide new studies towards on automating CR assignment.

  • [PDF] [DOI] Y. C. Cavalcanti, P. A. D. M. S. Neto, I. D. C. Machado, E. S. de Almeida, and S. R. L. de Meira, “Towards Understanding Software Change Request Assignment: a survey with practitioners,” in Proceedings of The 17th International Conference on Evaluation and Assessment in Software Engineering (EASE’2013), 2013, pp. 195-206.
    [Bibtex]
    @INPROCEEDINGS{CavalcantiEASE2013,
    title = {{Towards Understanding Software Change Request Assignment: a survey with practitioners}},
    author = {Yguaratã Cerqueira Cavalcanti and Paulo Anselmo Da Mota Silveira Neto and Ivan Do Carmo Machado and Eduardo Santana de Almeida and Silvio Romero de Lemos Meira},
    booktitle = {{Proceedings of The 17th International Conference on Evaluation and Assessment in Software Engineering (EASE'2013)}},
    abstract = {
    Context: Change Request (CR) repositories play an important role in the software maintenance and evolution process. Through a CR repository, software changes are reported and assigned to developers. Finding the appropriate developer to a CR is crucial for obtaining the lowest, economically feasible, fixing time. Nevertheless, assigning CRs is a labor-intensive and time consuming task. Although many work have proposed automated approaches for CR assignment, they have been implemented without investigating the fundamental aspects which characterize the task itself.
    Objective: This paper investigates the impact of CR assignment on developers productivity and identifies the fundamental aspects that characterize it, such as the strategies to perform the assignments and the complexity involved in them. Such investigation improves the current knowledge on the topic, providing researchers and practitioners with useful information towards developing effective solutions.
    Method: A survey was performed with software developers to understand CR assignment in the Brazilian Federal Organization for Data Processing. The questionnaire was composed of 38 questions, being them both open-ended and closed-ended. We analyzed the answers of 36 respondents.
    Results: We find that: there is a significant amount of time being spent on assignments (e.g., assigning 20 CRs can take up to 3.3 hours); there are many strategies used to assign CRs, which are complementary to those used in current automated solutions; and CR assignment is very complexity due to a process that requires cognitive abilities for information seeking, communication, and memorization.
    Conclusion: CR assignments are fundamental to software maintenance, however it is an expensive activity. Although we understand that fully and totally accurate automation is unlikely, further improvements on this direction are feasible and necessary to reduce costs. This way, this paper brings relevant findings to guide new studies towards on automating CR assignment.},
    keywords = {Software Maintenance, Change Request Assignment, Bug Triage, Bug Tracking, Issue Tracking},
    year = {2013},
    month = {April},
    pages = {195-206},
    isbn = {978-1-4503-1848-8},
    doi = {10.1145/2460999.2461028},
    }
Share

The Lack of Rigour on Software Engineering Research

Last issue of ACM SIGSOFT Software Engineering Notes (valume 37, issue 5), Panos Louridas and Georgios Gousios published a very interesting paper: “A Note on Rigour and Replicability”. They selected a research paper from ICSE’2012 as a counter-example of how to make a research experiment and publish it ready for replications. They analyzed such a paper and commented on every single point of lack of information about the reported experiment. In summary, that paper have been published in a famous Software Engineering conference with few information regarding to the experiment, which makes impossible a further replication.

In research context, without experiment replications, especially in different contexts, results’ generalization and knowledge building become hard (not to say that it is almost impossible). Moreover, with this lack of information on the experiment, it is even hard to compare results among similar experiments. In my Ph.D. research I need to analyze several research papers on mining software repositories, specially Change Request repositories (i.e., Bugzilla and Mantis). I have to admit that most of the papers I’ve read suffer with this issue. I can say that in none of them it is possible to replicate the findings or reuse the method for other context. Of course, it is necessary to give a discount to research field on Software Engineering since it is not mature as social sciences, for instance. Actually, we need to learn with these other more mature sciences.

On the other hand, Louridas and Gousios pointed to what I believe to be the right way to overcome this situation. They argued that Software Engineering conferences should require all necessary data and information for the experiment replication of accepted papers (or under review), such as: measurement data, interviews, questionnaires, research protocols, scripts of statistical programs (i.e., R or SPSS), and so on. Additionally, I believe that in first place it is also necessary to ensure that the reviewers and researchers are aware of a common protocol that should be followed for writing an experimental research paper. That is, both of them should ask to themselves: Does the paper provide enough information for the experiment replication? If all of these points were considered, then we can expect a significant advance on Software Engineering research.

Finally, for those who want to improve their experiment papers I suggest to take a look in the references pointed in [1]. But do not be restricted to Software Engineering authors, also look for papers and books from social sciences research. Although medicine research is also a good source of information, in the context of experiments I believe that we are more close to the social sciences than we think. Indeed, apart the technology aspects of Software Engineering, it has a lot to do with human behavior. Thus, as a start point I would suggest the free book A Guide to Quantitative and Qualitative Dissertation Research [2].

References

[1] Louridas, P., & Gousios, G. (2012). A note on rigour and replicability. ACM SIGSOFT Software Engineering Notes, 37(5), 1. doi:10.1145/2347696.2347706

[2] Jr, J. S. (2012). A guide to quantitative and qualitative dissertation research. Retrieved from http://diginole.lib.fsu.edu/edpsy_faculty_publications/1/

Share

An Experimental Study to Evaluate a SPL Architecture Regression Testing Approach – IRI 2012

In Software Product Line Engineering, where products are derived from a common platform, the reference architecture should be considered the main asset. In order to maintain its correctness and reliability after modifications, a regression testing approach based on architecture specification and code was developed. In this paper, we evaluate it in two different scenarios, the corrective scenario, in which it was performed after a corrective change in the code, and in the progressive scenario, executed after a evolution or enhancement, when some specification changes. The results showed that the progressive scenario was less costly than the corrective. Our evaluation also highlights the importance of a code-based technique to select a set of efficient and effective test cases.

  • [DOI] P. A. M. S. da Neto, I. D. C. Machado, E. S. de Almeida, Y. C. Cavalcanti, and V. C. Garcia, “An Experimental Study to Evaluate a SPL Architecture Regression Testing Approach,” in Proceedings of the 13th ieee international conference on information reuse and integration (iri’2012), 2012, pp. 608-615.
    [Bibtex]
    @INPROCEEDINGS{SilveiraNeto2012,
    title = {{An Experimental Study to Evaluate a SPL Architecture Regression Testing Approach}},
    author = {Paulo Anselmo da Mota Silveira Neto and Ivan Do Carmo Machado and Eduardo Santana de Almeida and Yguaratã Cerqueira Cavalcanti and
    Vinicius Cardoso Garcia},
    booktitle = {Proceedings of The 13th IEEE International Conference on Information Reuse and Integration (IRI'2012)},
    abstract = {In Software Product Line Engineering, where products are derived from a common platform, the reference
    architecture should be considered the main asset. In order to maintain its correctness and reliability after modifications, a regression testing approach based on architecture specification and code was developed. In this paper, we evaluate it in two different scenarios, the corrective scenario, in which it was performed after a corrective change in the code, and in the progressive scenario, executed after a evolution or enhancement, when some specification changes. The results showed that the progressive scenario was less costly than the corrective. Our evaluation also highlights the importance of a code-based technique to select a set of efficient and effective test cases.},
    keywords = {Software Testing, Software Product Lines, Regression Testing},
    year = {2012},
    month = {August},
    doi = {10.1109/IRI.2012.6303065},
    pages = {608--615},
    }
Share