Understanding, Modelling, and Reasoning about Software

Empirical and Formal software engineering methods are increasingly used synergistically to make the creation of software-based systems faster and safer. This course introduces students to key emerging techniques and technologies for Empirical SE and Formal Methods. The goal is to present a synthesis of recent developments in software engineering research and practice and identify opportunities for practical, evidence-based improvements to SE practice.

Quick links:

Course specifics

Instructor: Michalis Famelis, office: AA2343, email: famelis@iro.umontreal.ca
Office hours: by appointment

Class times: Tuesdays and Thursdays, 09:30 – 11:30

Please note the changing meeting venues for the next two weeks:

  • Tuesday, January 10th: Z-209 Pav. Claire-McNicoll
  • Thursday, January 12th: 1409 Pav. André-Aisenstadt
  • Tuesday, January 17th: Z-215 Pav. Claire-McNicoll
  • Thursday, January 19th: 3195 Pav. Andre-Aisenstadt

Relevant pages:

The course (lectures, material, and student deliverables) will be in English.

Marking scheme

Paper presentation: 30%
Discussion and participation: 10%
Paper reviews: 20%
Project: 40%

Students are expected to read on average two papers each week, in advance of class.

During “Paper discussion” sessions, one student will present a paper, for which the other students must prepare (short) written reviews, which will be used as the basis for further discussion.

See also:

Paper reviews are due before class on the day the paper is being presented.

Students will work on term projects, jointly defined with the instructor. See the course schedule for timetable and deliverables.

See also:

Course timetable

Week Date Theme   Topic   Deliverable
1 10-Jan Introduction
12-Jan Modelling and specification   Models and metamodels
2 17-Jan Formal specification
19-Jan Flexible modelling; Reverse engineering
3 24-Jan Paper discussion
26-Jan Paper discussion
4 31-Jan Reasoning and analysis   Logic and reasoning
02-Feb Modelling and analysis
5 07-Feb Reasoning over sets
09-Feb Model checking Project description
6 14-Feb Paper discussion
16-Feb   Paper discussion
7 21-Feb Discussion about projects Project plan
23-Feb Empirical methods   Approaches to epistemology
8 28-Feb No class
02-Mar No class
9 07-Mar Empirical methods
09-Mar Choosing empirical methods
10 14-Mar Paper discussion
16-Mar   Paper discussion
11 21-Mar Discussion about projects Project milestone
23-Mar Software engineering practise   Software engineering processes
12 28-Mar Model-driven development
30-Mar Agile and DevOps
13 04-Apr Paper discussion
06-Apr   Paper discussion
14 11-Apr Project presentations
13-Apr Project presentations
15 20-Apr Project due

Paper presentations

List of readings

Each student will present four papers over the course of the term. We will devote 50 minutes of class time to each paper. The presenter will first give a 20-minute conference style presentation of the paper, and then will lead the class in a discussion of the paper’s contributions.

Paper Presentation Requests

Your paper presentation requests are due on Wednesday, January 18, 2017. Send email to famelis@iro.umontreal.ca, with the subject: IFT6251 Paper Selections.

The message body should include:

  • your name
  • your preferred e-mail address
  • titles of papers from the reading list that you would prefer to present or review. For each of the modules of the course (A,B,C,D), choose up to 3 papers and prioritize your choices (you can include up to two papers not on the reading list – they may or may not be presented during the course)

I will try to have paper assignments on-line by January 19.

Presentation Information

When preparing your presentation, keep in mind that the audience will have already read the paper that you are presenting. Also, some general introductory topics may have already been covered in lectures or in previous presentations. Your presentation should:

  • remind your audience of what problem is addressed in the paper and why that problem is important
  • identify what you feel are the most significant and important observations, ideas, and results in the paper,
  • put the paper in context of the course
  • raise issues for discussion

The slides for your presentation are due by 9:00am on the day you are presenting. Email the instructor a single Powerpoint or PDF file containing your slides.

The slides will be posted on StudiUM.

All students (not just the presenter!) are expected participate in the discussion of your paper. Come to class prepared with some questions for the class and comments.

Marking

Paper presentations are worth 30% of your final grade. Presentations will be evaluated according to the following criteria:

  • Organization: talk includes an introduction, middle section, and conclusions; parts of the presentation follow from each other; topic(s) placed in the context of the course; problem and solution clearly identified; evaluation of the solution included; presentation is appropriately aimed at audience; key points of the presentation summarized at the end.

  • Presentation ability: well rehearsed; speaks clearly, audibly, and not too fast; makes eye contact with the class; mannerisms suitable; finishes within allotted time; enthusiasm in voice; smooth speech.

  • Discussion and Knowledge: the talk stimulates interesting questions; the presenter poses interesting questions, presenter is familiar with the paper’s content; able to answer questions about the material during discussion period.

For each criterion, you will receive a mark between 1 and 5:
1: Poor, 2: Needs Work, 3: Satisfactory, 4: Very Good, 5: Outstanding

The formula for computing your paper presentation mark is: x * 2 + 70 where x is the mark out of 15. Thus, it is impossible to get below 70% for a presentation (unless you don’t show up!).

Paper reviews

Paper review form

Process

During “Paper presentation” sessions, one student presents a paper and the rest must prepare a review, which will be used as the basis of further discussion. There is a plain text review form (available via the above link) which should be used for all reviews.

When completing your review, pay particular attention to the section that asks you to summarize the paper’s contributions and to justify your rating of the paper. State what you consider to be the main contributions of the paper, and summarize your rationale for your ratings. Keep the whole thing short: two short paragraphs is the upper limit. Think of this section as an “executive summary” of both the paper and your review of the paper. Any additional detail belongs in the final section of the review, which can be as long and as detailed as necessary.

Reviews are due by 8:00am on the day in which the corresponding paper is being discussed in class. Email your completed review to famelis@iro.umontreal.ca using as subject: “IFT6251 Paper Review” and with the review as a plain text file attachment. Do not cut-and-paste or otherwise in-line your review into the body of your message.

Marking

Paper reviews are worth 20% of your final grade. Presentations will be evaluated according to the following criteria:

  • Summary: the summary accurately reflects the content of the paper, makes it obvious that the student read the paper

  • Clarity: the review is understandable and clearly written

  • Validity: the ratings for the paper follow from the list of strengths/weaknesses identified by the student

  • Non-triviality: the review does more than scratching the surface of the paper

For each criterion, you will receive a mark between 1 and 5:
1: Poor, 2: Needs Work, 3: Satisfactory, 4: Very Good, 5: Outstanding

The formula for computing your paper review mark is: x * 4 + 20 where x is the mark out of 20. Thus, it is impossible to get below 20% for a presentation (unless you don’t submit one on time).

Course projects

The course project is worth 40% of your final grade. Several kinds of projects can be carried out. The deliverables are:

  • A project description, due February 9th.

  • A detailed project plan, due February 21st.

  • At the completion of the project on April 20th:

    • In-class project presentation and/or demo
    • A written report, if applicable
    • A copy of any software developed as part of your project, with appropriate documentation and with a short (1 page) usage guide
    • A copy of all empirical data collected, if applicable

Evaluation

Project marks will depend on the depth and novelty of the ideas embodied in your project, on the quality of your methodology, and on the quality of your written report, in-class presentation, and software (if applicable).

Project topics

Students are encouraged to propose and discuss their own project topics. Ideally, the projects should include an empirical and a formal component.

Some preliminary “seed” ideas:

  1. Plan an empirical study, and configure and extend a model management platform (MMINT or Epsilon), demonstrating how it can be used as the experimental platform.

  2. Creation of a non-trivial model of a real system in formal modeling language (e.g., Alloy), development/implementation of some analysis technique.

  3. Information extraction, reverse engineering and generation of analytics form open source projects (e.g. on GitHub).

  4. Identification of design-related artifacts in conversations between developers.

Readings

[A] Modelling and specification

  1. Harel, D, Rumpe, B. Meaningful Modeling: What’s the Semantics of “Semantics”?, IEEE Computer, 2004

  2. Daniel Jackson. 2002. Alloy: a lightweight object modelling notation. ACM Trans. Softw. Eng. Methodol. 11, 2 (April 2002), 256-290.

  3. Kang E, Jackson E, Schulte W. An approach for effective design space exploration. In Monterey Workshop 2010 Mar 31 (pp. 33-54). Springer Berlin Heidelberg.

  4. Michael Jackson and Pamela Zave. 1995. Deriving specifications from requirements: an example. In Proceedings of ICSE’95.

  5. Robert Allen and David Garlan. 1994. Formalizing architectural connection. In Proceedings of ICSE’94

  6. Crane, M.L. and Dingel, J., 2005, October. UML vs. classical vs. Rhapsody statecharts: Not all models are created equal. In Proceedings of MODELS’05.


[B] Reasoning and analysis

  1. H. Goldsby, B. Cheng. Automatically Discovering Properties that Specify the Latent Behavior of UML Models. In Proceedings of MoDELS’10, 2010

  2. S. Maoz, J.O. Ringert, B. Rumpe. CD2Alloy: Class Diagrams Analysis Using Alloy Revisited. In Proceedings of MoDELS’11, 2011

  3. M. Famelis, R. Salay and M. Chechik, Partial models: Towards modeling and reasoning with uncertainty, 2012 34th International Conference on Software Engineering (ICSE), Zurich, 2012, pp. 573-583.

  4. Classen, A., Heymans, P., Schobbens, P. Y., Legay, A., & Raskin, J. F. (2010, May). Model checking lots of systems: efficient verification of temporal properties in software product lines. In Proceedings of ICSE’10.

  5. Beschastnikh, I., Brun, Y., Schneider, S., Sloan, M., & Ernst, M. D. (2011, September). Leveraging existing instrumentation to automatically infer invariant-constrained models. In Proceedings of the FSE’11.

  6. Aleksandar Milicevic, Joseph P. Near, Eunsuk Kang, and Daniel Jackson. 2015. Alloy*: a general-purpose higher-order relational constraint solver. In Proceedings of ICSE 2015.


[C] Empirical methods

  1. Thomas Zimmermann, Peter Weisgerber, Stephan Diehl, and Andreas Zeller. 2004. Mining Version Histories to Guide Software Changes. In Proceedings of ICSE’04, 2004

  2. Regina Hebig, Truong Ho Quang, Michel R. V. Chaudron, Gregorio Robles, and Miguel Angel Fernandez. 2016. The quest for open source projects that use UML: mining GitHub. In Proceedings of MODELS’16, 2016

  3. Michael Szvetits and Uwe Zdun. 2016. Controlled experiment on the comprehension of runtime phenomena using models created at design time. In Proceedings of MODELS’16, 2016

  4. Dubinsky, Y., Rubin, J., Berger, T., Duszynski, S., Becker, M., & Czarnecki, K. (2013, March). An exploratory study of cloning in industrial software product lines. In Software Maintenance and Reengineering (CSMR), 2013 17th European Conference on (pp. 25-34). IEEE.

  5. Mathieu Lavallée and Pierre N. Robillard. 2015. Why good developers write bad code: an observational case study of the impacts of organizational factors on software quality. In Proceedings of ICSE’15.

  6. Petre, M., 2014. “No shit” or “Oh, shit!”: responses to observations on the use of UML in professional practice. Software & Systems Modeling, 13(4), pp.1225-1235.


[D] Software engineering practise

  1. L. Osterweil. 1987. Software processes are software too. In Proceedings of ICSE’87.

  2. Davor Čubranić and Gail C. Murphy. 2003. Hipikat: recommending pertinent software development artifacts. In Proceedings of ICSE’03, 2003.

  3. Audris Mockus, Roy T. Fielding, and James Herbsleb. 2000. A case study of open source software development: the Apache server. In Proceedings of ICSE’00, 2000.

  4. Hamed Esfahani, Jonas Fietz, Qi Ke, Alexei Kolomiets, Erica Lan, Erik Mavrinac, Wolfram Schulte, Newton Sanches, and Srikanth Kandula. 2016. CloudBuild: Microsoft’s distributed and caching build service. In Proceedings of ICSE’16

  5. Xiao He, Paris Avgeriou, Peng Liang, and Zengyang Li. 2016. Technical debt in MDE: a case study on GMF/EMF-based projects. In Proceedings of MODELS’16, 2016

  6. Fahad R. Golra, Antoine Beugnard, Fabien Dagnat, Sylvain Guerin, and Christophe Guychard. 2016. Using free modeling as an agile method for developing domain specific modeling languages. In Proceedings of MODELS’16, 2016