|
| 07:30 am |
Registration |
|
Welcome Session |
| 08:30 am |
Introduction & Speed Dating |
|
Session 1 (Foutse Khomh and Bram Adams) |
|
08:50 am |
Invited Talk by Pete Rotella
(Quality Engineer at Cisco Systems, USA)
Bio
Pete Rotella works in the Corporate Quality Group at Cisco Systems in Research Triangle Park, NC, USA. His areas of interest include using in-process metrics to help improve engineering practices, and predicting customer-found defect levels for various types of software releases. Prior to Cisco, he developed statistical models and tools, and other types of software systems, at several Fortune 100 companies, startups, and government agencies. |
| 09:45 am |
Predicting Software Field Reliability by Pete Rotella, Sunita Chulani and Devesh Goyal (Cisco Systems, Inc.) [research talk]
Abstract:
The objective of the work described is to accurately predict, as early as possible in the software lifecycle, how reliably a new software release will behave in the field. The initiative is based on a set of innovative mathematical models that have consistently shown a high correlation between key in-process metrics and our primary customer experience metric, SWDPMH (Software Defects per Million Hours [usage] per Month). We have focused on the three primary dimensions of testing – incoming, fixed, and backlog bugs. All of the key predictive metrics described here are empirically-derived, and in specific quantitative terms have not previously been documented in the software engineering/quality literature.
Performance of defect prediction in rapidly evolving software by Davide Giacomo Cavezza, Roberto Pietrantuono and Stefano Russo (Universita degli Studi di Napoli Federico II) [research talk]
Abstract:
Defect prediction techniques allow spotting modules
(or commits) likely to contain (introduce) a defect by training models with product or process metrics – thus supporting testing, code integration, and release decisions. When applied to processes where software changes rapidly, conventional techniques might
fail, as trained models are not thought to evolve along with the software.
In this study, we analyze the performance of defect prediction in rapidly evolving software. Framed in a high commit frequency context, we set up an approach to continuously refine prediction models by using new commit data, and predict whether or not an attempted commit is going to introduce a bug and to issue a
warning in this case. An experiment is set up on the Eclipse JDT software to assess the prediction ability trend. Results enable to leverage defect prediction potentials in modern development paradigms with short release cycle and high code variability.
Discussion |
| 10:30 am |
Coffee Break (with poster session) |
|
Session 2 (Hyrum Wright and Shane McIntosh) |
| 11:00 am |
Continuous Deployment and Schema Evolution in SQL Databases by Michael de Jong and Arie van Deursen (Delft University of Technology) [research talk]
Abstract: Continuous Deployment is an important enabler of rapid delivery of business value and early end user feedback. While frequent code deployment is well understood, the impact of frequent change on persistent data is less understood and supported. SQL schema evolutions in particular can make it expensive to deploy a new version, and may even lead to downtime if schema changes can only be applied by blocking operations.
In this paper we study the problem of continuous deployment in the presence of database schema evolution in more detail. We identify a number of shortcomings to existing solutions and tools, mostly related to avoidable downtime and support for foreign keys. We propose a novel approach to address these problems, and provide an open source implementation. Initial evaluation suggests the approach is effective and sufficiently efficient.
Securing a Deployment Pipeline by Len Bass, Ralph Holz, Paul Rimba, An Binh Tran and Liming Zhu (NICTA, Australia) [research talk]
Abstract: At the RELENG 2014 Q&A, the question was asked "What is your greatest concern?" and the response was "someone subverting our deployment pipeline". That is the motivation for this paper. We address several questions:
-What does it mean to subvert a pipeline? We provide several different scenarios of subversion.
-How does one secure a pipeline? We provide an engineering process that is based on having trusted components mediate access to sensitive portions of the pipeline from other components, which can remain untrusted.
Applying our process to a pipeline we constructed involving Chef, Jenkins, Docker, Github, and AWS, we find that some aspects of our process result in easy to make changes to the pipeline, whereas others are more difficult. Consequently, we have developed a design that hardens the pipeline, although it does not yet completely secure it.
Discussion
|
| 11:45 am |
Extracting Configuration Knowledge from Build Files with Symbolic Analysis by Shurui Zhou, Jafar Al-Kofahi, Tien N. Nguyen, Christian Kaestner and Sarah Nadi (Carnegie Mellon University, Iowa State University, and Technische Universitaet Darmstadt) [research talk]
Abstract:
Build systems contain a lot of configuration knowledge about a software system, such as under which conditions specific files are compiled. Extracting such configuration knowledge is important for many tools analyzing highly-configurable systems, but very challenging due to the complex nature of build systems. We design an approach, based on SYMake, that symbolically evaluates Makefiles and extracts configuration knowledge in terms of file presence conditions. We implement an initial prototype and demonstrate feasibility on small examples.
Continuous Delivery with Jenkins by Valentina Armenise (Cloudbees) [practitioner talk]
Abstract:
This paper illustrates how Jenkins evolved from being a pure Continuous Integration Platform to a Continuous Delivery one, embracing the new design tendency where not only the build but also the release and the delivery process of the product is automated.
In this scenario Jenkins becomes the orchestrator tool for all the teams/roles involved in the software lifecycle, thanks to which Development, Quality&Assurance and Operations teams can work closely together.
Abstract: Continuous delivery (CD) of modularized products have given Tasktop flexibility to break free from a rigid component release schedule dictated by product releases. However, this flexibility poses several challenges for continuous integration, dependency management, developer productivity, release planning, and build repeatability. These challenges are amplified by a quickly growing product list which share many common components that are written in both Java and JavaScript. Our growing product list includes 36 product distributions composed of five products, two partner products, Visual Studio plugins, Eclipse plugins and a Connector SDK. This mix of products has lead to a fairly large toolset that includes Maven Tycho builds, Gerrit/Git, Jenkins, Maven repositories, P2 repositories, NodeJS, Grunt, Bower repositories, Node Package Manager (NPM), Nullsoft Scriptable Install System (NSIS installer), and IzPack.
I will discuss how Tasktop can balance the inherent advantages CD without increasing the complexity of our development and release process to the point that it affects productivity and product quality. This is especially important as we move to enable development teams to design, build, test and release components independently in an internal open source software development model. This talk will also touch on the challenges of maintaining legacy build systems, release branches and infrastructure to preserve build repeatability for the duration of our eight year product support timeline.
Discussion |
| 12:30pm |
Lunch & Posters |
|
Session 3 (Sarah Nadi and Peter C. Rigby) |
|
02:00 pm |
Collecting Release Metadata at Google by Dominic Mitchell (Google UK) [practitioner talk]
Abstract:
This talk will describe the release event logger system, used for collecting release metadata at Google. With over 5000 projects making new releases weekly (or more frequently), it's important to be able to keep track of who did what. All our release tools automatically log metadata into a centralised service. The logs produced are invaluable for understanding not only individual releases and their constituent events, but also for gaining insight into historical actions.
Continuous Delivery in a Financial Organization by Chris Bartels and Daniele Romano (ING Netherlands) [practitioner talk]
Abstract: After moving to Agile methodologies the next step in innovating ING IT landscape has been Continuous Delivery. Currently more than 250 software systems can go to production from commit in 15 minutes through a complete automated Continuous Delivery pipeline. The ING pipeline includes Backlog Management, Continuous Integration, Agile Testing, Quality Assurance, Automatic Deployment, and Production Readiness control systems. Every software system goes through the same Continuous Delivery pipeline that the DevOps teams can control and configure to adhere their needs. The continuous delivery pipeline allowed DevOps teams to reduce their cycle time from more than 120 days to approximately 20 days in less than one year. Discussion |
| 02:45 pm |
Research Opportunities in Continuous Delivery - Reflections from Two Years' Experiences in A Large Bookmaking Company by Lianping Chen (Paddy Power) [practitioner talk]
Abstract: We have been implementing continuous delivery in Paddy Power, a large organization in the bookmaking industry, for more than two years. In this talk, I will reflect on our journey to continuous delivery and discuss the research opportunities I see.
Towards Uniform Definitions for Release Engineering and DevOps by Ralf Penners, Andrej Dyck and Horst Lichter (RWTH Aachen University) [research talk]
Abstract:
Delivering software as fast as possible is essential for software development organizations, in order to keep in pace with competitors. Accordingly, release engineering, a software engineering discipline concerned with the delivery and deployment process, plays an important role in such organizations. Further, in recent years, the term DevOps gained popularity in the IT world. It describes an approach to improve the collaboration between development and IT operations teams, in order to streamline software engineering processes.
To the best of our knowledge, there is no uniform definition for neither of these terms, and thus, many inadequate or even wrong definitions exist. Consequently, these terms are often confused or even used as synonyms. In this paper, we will tell those two terms apart by contrasting available definitions and descriptions for both of them. Additionally, we propose uniform definitions for release engineering and DevOps, which we developed in cooperation with experts.
Discussion
|
| 03:30 pm |
Coffee Break (with poster session) |
|
Session 4 (Foutse Khomh and Bram Adams) |
| 04:00 pm |
RelEng as a force multiplier by John O'Duinn (Release Mechanix) [Practitioner talk]
Abstract:
When done well, Release Engineering can be a force multiplier that
enables software companies to ship software efficiently and reliably, to
grow in the marketplace, to grow internally by hiring more developers,
and to compete against other above-their-weight companies. Done right, this can also make the organization a more humane place to work!
When done badly, larger companies with bigger budgets and bigger staff can find themselves being out-manoevered by smaller, more nimble competitors... or find themselves unable to handle changing market dynamics quickly because of the inefficiencies in their organization.
To have this culture change happen, its important that everyone understands the value, and everyone understands how they play a role in making the organization efficient. This means RelEng needs to be clearly explained in non-Release-Engineer language, to all portions of the company. Organizational culture change is hard - I'll describe some approaches that have, and haven't, worked for me over the last 23 years of my career in Release Engineering, and how things played out in several different case studies.
Note: While I had the same *title* for the RelEngConf 2013 keynote, this is not a repeat of the same talk. I just think the title summarizes the talk, is easy to translate and is easy to remember.
|
| 04:45 pm |
Discussion & Wrap-Up |
| 05:30 pm |
End of Workshop and Leaving for Workshop Dinner (directions here) |
Accepted Poster:
|
