Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Choosing the right software metrics is not an easy task. If you have access to data, and in software development a lot of data can be gathered, you can easily think of many possible metrics. Sometimes, too many... up to the point of reaching saturation!
This report aims to formalize a strategy and a method to identify, acquire and understand metrics, and how to apply those to the inner source world. Part of the information found in this document is built on top of previous literature and talks about InnerSource. However, none of the former seems to focus specifically on the metrics needed to understand if a process is working as expected, or if substantial changes are required.
As InnerSource is a medium or long term activity, as any new methodology to be applied within a company, feedback about the process and performance is key to assure its quality. This report has as goal to be the glue between developers, managers and C-level. It aims to help them build a proper mining software structure based on key indicators that will help to lead the improvement of the process. And specifically in the case of inner sourcing, this document is expected to help when selecting initial metrics and studies to lead that process.
It is worth mentioning that the whole organizational structure should be aligned with the metrics used for the analysis. Qualitative feedback from the several layers of the organization is also a key part of this process, involving from developers to C-level and going through middle management. They all should understand that these tracking actions are following a specific goal and not tracking their own individual activities.
Rewarding systems on top of the metrics are also recommended, but always with a specific focus on fostering some actions such as pushing developers to commit their first pull request[^1]. The point about having metrics is that people can cheat on them, so when fostering specific behaviors, those should be use in short periods of time to help developers to get used to some way of developing. The most recommended use of metrics is to track the performance of the whole community and how to avoid bottlenecks and actions that may delay their activity.
The approach presented in this report follows the GQM approach (Goal-Question-Metric)[^2]. This consists of declaring the goal(s) of a specific decision, then state a set of questions that fit in that goal, and finally come back with a list of metrics that answer each of the proposed questions.
As InnerSource has a different meaning depending on the organization where it is deployed, InnerSource may have different goals. However, there are some goals that are usually required across organizations and that can be divided into these main groups[^3]:
Improve code quality through continuous integration (CI). Open source projects are used to CI and peer review process. And specifically having many eyes to any piece of code going to master help a lot to detect potential issues in advance. It has been measured that having code review in your process helps up to the point of saving half of the potential spending when maintaining the software[^4]. On the other hand, CI allows to automate checks that were usually done by human beings, such as unit testing, regression tests, style checkers and others.
Decrease time to market. As silos are broken within the organization, there are developers and business units across the organization that are willing to work on the same topic. And this helps to increase the velocity of the development process.
Allow innovation within developers. Developers are now allowed to collaborate with others and create their own social networks with other business units. This collaboration helps to bring new points of view to the same problem what allows to bring innovation to the same problem. Developers can easily now create new repositories under some rules, having those projects as incubators and get traction from other developers that may participate in such project at some point. Do-ocracy could be also part of the process.
Reduce development and maintenance costs. Maintenance is reduced to the decrease of business units working on the same topic and this cost is now shared among all of them. And this applies to the development and maintenance point of view. Any business unit interested in participating in a project as their goals are aligned to such project may provide resources.
And last but not least, allowing developers to work on the topics they feel are important for the organization or on their own interest help them to be more comfortable. InnerSource helps to improve the retention of the good developers for the organizations, but also helps in the recruitment process as the organization is seen as innovative and listen to developers needs. This helps to the general engagement in the organization.
An overview of the Goals, Questions, and Metrics (GQM) catalog.
Right-click on any node representing a goal, question, or metric to open a new tab with more detailed information.
Add your goals, questions, and metrics into this graph! It will help you to see how others approach and interact with what you are doing. You may get new ideas of what metrics answer the questions you have or what additional goals your questions can support. See CONTRIBUTING.md#metrics.
Why should be build things more than once? We're already busy and behind where we'd like to be. We want to build just once software that solves common problems and then share it widely throughout the company.
Synopsis: Count how many times the InnerSource project is used. Gain additional insight by gathering metadata on these usages such as business unit or time frame. With this additional metadata we can see how lopsided or even the usage is spread across the company and across time.
Unit of Measurement: Ordinal number.
Interpretation: Higher absolute usage count is generally better. Higher diversity of usage across business unit or time frame is generally better.
Measuring
If the project is an API and requires caller authentication, then read the API logs to determine the list of callers.
If the project is a module (or package), then scan your source control system for files that list out dependencies for a package manager to install. Here is a list of . Look in those dependencies for usage of your InnerSource project.
The project will parse out these dependencies for you.
is a security tool that your organization may already use. As part of its operation, it already parses out these dependencies. This information is exposed via .
Question
How it relates to the goal
Gotchas
Every instance of reduced duplication will show up as an increase in usage.
The converse is not necessarily true that every instance of increased usage is an instance of reduced duplication. It's always possible that some shared usage would have happened even without InnerSource.
The Goal Question Metric approach defines a mechanism to provide an umbrella that helps to define and detail software metrics. This can be easily extended to other backgrounds, although InnerSource method basically fits in this type of evaluation.
As stated in the original paper: “Any engineering process requires feedback and evaluation. Software development is an engineering discipline and measurement is an ideal mechanism for feedback and evaluation. The measurement and information... helps in the understanding and control of the software processes and products. It helps in making intelligent decision and improving over time. But measurement must be focused, based upon goals and models. We need to establish goals for the various software processes and products and these goals should be measurable, driven by the appropriate models.”[^6]. The GQM has been extensively used in academia and the industry as a method to detail metrics useful for a set of pre-defined goals.
In the specific case of InnerSource, and summarizing, the following can be seen as some of the most important goals for InnerSource:
Goal 1. Improve code quality through CI and peer review
Goal 2. Decrease time to market
Goal 3. Allow innovation within developers
Goal 4. Reduce development and maintenance costs
Goal 5. Improve engagement within the organization
Then, each of the goals may have one or more questions that will answer those specific goals. And those are particularly related to the stage the process is. Although we may have the same goal when starting to apply InnerSource within an organization and when that method is fully established, the related questions are potentially different.
Let’s focus on Goal 5: improve engagement within the organization. We may start this process from a hierarchical organization where silos of developers exist and agile is used as their main methodology for software development. The specific question for a starting situation would be to understand if specific policies, such as opening the infrastructure, allowing public discussion, having design meetings and a long tail of activities fostering interactions among developers are actually working. On the other hand, in an inner-sourced organization, the question would be to keep improving the engagement or at least have a stable software development process with this respect.
Starting from scratch: Question 1: How many new attracted developers are participating in the InnerSourced projects?
Working on an inner-sourced community: Question 1: Is the attraction of new developers improving over time? And this question leads to Question 2: Are we retaining the new developers? And this finally leads to Question 3: How long does it take for a developer to leave the community?
It is clear that starting from scratch needs to focus on the initial stages of the process improvement, while in a mature process, there are other questions of importance.
And finally metrics. These are linked to each of the questions and may answer some of them at the same time. If we go for the four questions, we may define the following metrics:
Q1. How many new attracted developers are participating in the inner sourced projects?
M1. Number of newcomers.
M2. Relative number of newcomers out of the total number of developers.
Q2. Is the attraction of new developers improving over time?
M1. Number of newcomers.
M2. Relative number of newcomers out of the total number of developers
M3. Slope of the tendency of newcomers coming to the project
Q3. Are we retaining the new developers?
M4. Number of developers still developing that entered during the last year.
M5. Relative number of retained developers out of the newcomers during the last year.
Q4. How long does it take for a developer to leave the community?
Median of the total time of all of the developers since their first commit till their last commit.
Although open source communities seem to be similar in some cases, there are peculiarities that define their actual idiosyncrasy. Details such as selecting the code review tool may mean a significant change in the way this is analyzed. This also takes place in InnerSource projects.
Infrastructure is indeed a key part of the InnerSource process as this may force developers to work in a specific way. As an example, the OpenStack community uses Gerrit and forces developers willing to commit a piece of code to use this. This means that the community is 100% sure they are code-reviewing any piece of code into the baseline.
Thus, having the proper infrastructure helps to follow a pre-defined process, but this also helps to be more metrics-friendly as there is a clear workflow followed by any developer within the organization.
On the other hand, if the workflow is not well defined or if the development infrastructure allows to have work-arounds, this may lead to situations where shortcuts are usual, and developers and managers may get frustrated at some point.
Thus, having in mind those differences between two communities, inner source is not safe of this peculiarities, and any organization faces a list of potential issues when installing infrastructure, governance model, financial system and so on.
The following is a list of usual studies of interest in organizations applying InnerSource methodologies. This list is not intended to be an exhaustive one, but examples of how to approach and measure specific issues related to the usual goals in the InnerSource world. Part of these analysis are experiences coming from the open source world that could be useful when talking about InnerSource communities. These are communities and engaging new developers, retaining those, code review processes and others are common goals.
This analysis is related to the centralization of the development and attracting and retaining new developers. When aiming at incrementing the number of participants in the InnerSource ecosystem, trusted committers and mentors are key in this new process. Trusted committers as they are the core reviewers of the project and well known developers. And those trusted committers may be the ones acting as mentors. Mentorship should help new developers to feel comfortable with the environment in the project, and for this, mentors will help new contributors in several ways: understand how to use the available infrastructure (Git repositories, mailing lists, code review process, etc). The mentor will provide pointers to the documentation and code guidelines. This role will also help in understanding how to contribute to that specific community. And finally, will review the piece of code pull-requested by a newcomer driving her around the needs for such change and potential requested updates to that piece of code.
And all of this information can be tracked. As InnerSource is fostering a transparent infrastructure to deal with all of the activities around the development of the product, those discussions are all open to anyone interested in. Thus, the mentorship analysis will help to understand who are those mentors, if those mentors are helping to reduce the time to review process as they give clear directions about how to proceed and the number of newcomers that come to the community. All of these are just examples of how to potentially track mentors activity and if such activity is actually helping in the InnerSource adoption within an organization.
This should be compared to previous stages of the software development process and improve the bottlenecks of activity when needed. A potential and usual issue raised in open source communities, but also within organizations is that the workload adequacy of the mentors and core reviewers is high and those are at some point the bottleneck in the code review process, what forces the community to bring more core reviewers.
It is also interesting to foster the creation of the mentors role even if they are not core reviewers of the community, as they can provide useful information to the newcomers. This role can be also seen as the community manager that facilitates the path of those interested in contributing.
The study of the development cycle is related to the reduction of the time to market. This could be seen as a way to determine if the policies applied are actually decreasing the total time to develop a new feature, fix a bug and other actions. The goal of this analysis consists of understanding the total time from user stories to the merge into the source code repository.
This is important to understand the time that usually takes a new requirement to be implemented going through the design phase, implementation, code review process, continuous integration and merging time. As SLAs are playing a role here, we can calculate how long it takes for a feature request to be part of the source code and then later, there is a time to deploy that functionality or fix that in production.
Some examples of this analysis could be the questions of how fast the requirements are implemented, for example the best 50% or 80% of them and if the community is able to reduce that time over time. It is also possible to split this process into its several phases such as feature request, backlog, developing process, code review process, continuous integration, merge into master, more continuous integration and later deployment in the customer. And thanks to this split, it is possible to look for bottlenecks.
As an example, let us consider the code review phase. There are two roles playing here: the submitter and the reviewer. And we can split the time that a reviewer and a submitter is spending on their specific tasks. Thus, there is a time waiting for a reviewer action (typically the code review action) and the time waiting for a submitter action (typically when the reviewer asked for some modifications of the pull request).
If it happens that the time waiting for a submitter action is too high, the community may need to emphasized training actions on those newcomers or developers. But if the time waiting for a reviewer action is the one increasing over time, then the community may have a look at some actions that will help to reduce that time. For example, they can vote other developers to be code reviewers, or being more precise in the requests than to the submitters.
This analysis is focused on understanding how long it takes for a developer in the InnerSource community to become a developer or a core reviewer. Communities can be seen as onions where there are several layers. Those purely acting as end users. Those users that when they find a bug, this is reported to the community. Those that report the bug and send a pull request. And finally the development team that could participate in the community occasionally, regularly or as core developer.
The usual proportion of these communities, at least in open source, are following the distribution of 80% of the activity is done by a 20% of the developers. It is expected that InnerSource communities follow the same distribution as the development process is open within the organization. Thus, the mentioned roles such as pure end users or core developers will be also found.
Coming back to the analysis, the contributors funnel analysis aims at understanding how much time it takes for a developer to become a contributor of the InnerSource project. From their first traces asking for a feature request in a mailing lists or opening a bug report, till the point in time when they commit or review their first piece of code.
Indeed the goal of this analysis is to bring to context how good the community manager, mentors and other roles are performing with this respect.
This also helps to understand what percentage of contributors or users are finally committing a piece of code, and what percentage of those become trusted committer after a while.
Attraction and retention of new members. As InnerSource method evolves within an organization, it is expected that developers will work on other projects not directly related to their business units. Those participating for their first time will be part of the attraction rate of such community. Those that are still contributing in that project will be part of the retention rate. And those leaving the community after a while will be part of the developers that were not retained.
The ideal goal of any project is to attract as many as possible, retain all of them so nobody leaves the community. But this does not take place. Turnover is an inevitable situation and projects, business units and organizations have to deal with it. However it is possible to measure the engagement of the community and how specific policies help to attract and retain more developers than others.
That attraction and retention rate could be affected by other variables such as the programming language, people in the project, rewarding process and others.
Some specific questions in this analysis are related to how comfortable developers are in the business unit or in the organization, and will help to improve the process to attract and retain the best of them. How good is the project or community retaining developers? How far is this community from the community performing the best? Do we know the reasons why developers left the community? And if so, are we applying some actions to let them know that they are precious for the community?
The more the metrics we have, the more understanding the project will have about it. This could be also seen as a way to measure neutrality and transparency within a project and across several business units and the organization in general.
This concept is linked to large organizations that have several development teams and those are geographically distributed. Developers in different silos do not know what others are doing and can not interact with other developers out of their own silo.
Silos are also an effect of hierarchical organizations where only management on the top of the silo what others are doing. This can be even seen in development teams in the same silo where project members depends on middle management and that roles control all of the information.
As silos are broken and the structure becomes flatter, the knowledge should be shared across the organization. Developers come to a new project, they contribute and they gain knowledge about how to proceed, the mission of such project, idiosyncrasy, infrastructure, etc. And Those interactions can be measured thanks to the traces left in the several data sources.
For instance, developers working in the same file, may have different knowledge of the library, but there is a link between those: that piece of source code. Those acting as mentors and those being mentored also builds their own specific social network.
Some specific metrics of interest here are those related to the analysis of how the knowledge becomes something popular in the sense of people contributing to the several repositories of information. As an example, the territoriality can be used to understand how lonely are developers when working. Areas of the code highly territorial may mean that no contributors are helping there. And we need to understand the reason for this: spaghetti code, complex functionalities that requires a depth understanding of what is being developed, etc.
Other ways to measure how the knowledge is flowing around the community is analyzing the orphaned code left by those leaving the project. Is someone maintaining that area of the code? Are there others in charge of such piece of code that previously participated? What are the usual areas of knowledge of the different core reviewers where they participated?
And finally, usual metrics such as betweenness may help to understand key players that have knowledge across several projects. Those linking two nets are those that help to move knowledge from one community to the other.
Silos lead to middle management as those are in charge of filtering and controlling that the developer teams reach some specific deadlines and goals. However, activities among teams are not fostered with a lack of dynamism that affects other areas of innovation in the company as people tend to focus on the work their being paid for, and not others work.
In addition, this role in organizations have reached their position in a hierarchical way and this is how this is expected to work for others willing so scale in the hierarchy.
However, InnerSource aims at fostering developers to developers relationships (D2D) and not manager to manager (M2M) relationships as this delays the development process and decision making at the technical level.
And this leads to pure scalability. When allowing D2D relationships from any place within the organization, middle management roles tend to re-convert themselves to community managers that foster that type of behaviors, bring hackathons at home and invite third parties to participate in their projects as they participate in others.
And having freedom and trust from the organization developers work at is a good complement to feel great at work. Indeed one of the main goals of any organization is to keep their own great developers at home, but also to attract the best developers around the world. This cultural change and way of working help to have developers aligned with the general strategy of the company.
Thus, there are some initial goals to be accomplish within the company: remove silos, reconvert middle management, scale development and have a great development team. Following the GQM approach, this allows us to work on the specific questions we need to answer with this respect.
This follows the detailed GQM approach where each goal may have several questions and each question may have one or more metrics that helps to understand the situation. This also assumes that we are in an organization that already applied InnerSource and wanted to check if the process is leading to an actual improvement in the several detailed goals.
Goal: allow innovation within developers
Question: How are developers interacting with other business units?
Rationale: innovation is a result of mixing several points of view when resolving issues. Developers from other disciplines or business units will bring ideas that may foster that innovation process.
Metric: Number of attracted developers in the several inner source projects
Metric: Number of retained developers in the several inner source projects
Metric: Number of contributions: commits, wiki editions, emails and others from attracted contributors
Metric: Number of trusted committers
Question: How are new projects being created within the organization?
Rationale: the creation of new projects is the result of allowing developers to feel comfortable within the organization. New projects are the consequence of letting them know that they can bring ideas to the core business of the organization.
Metric: Number of new projects in the platform
Metric: Number of different developers creating new projects
Metric: Number of active developers in the last
Question: Are incubation projects gaining traction within the organization?
Rationale: As incubated projects need some process to become mature ones, this is related to the activity in the incubation side of the InnerSource program.
Metric: Number of new incubating projects
Metric: Number of incubated projects became official ones
Question: How many new technologies have been introduced in the organization?
Rationale: it is representative the PayPal case[^7] where used to work in a specific programming language. But innovation came from new developers introducing JS and saving hundreds of lines of code.
Metric: Number of new programming languages
Metric: Number of new open source libraries introduced in the development
This section aims at providing a strategy for your InnerSource metrics that help to understand the path from your initial process till a full InnerSource organization.
It is important to remark that metrics useful for some organizations are not that useful in other contexts. This is similar to the open source projects where a project is not that similar to other in terms of governance, licenses, infrastructure or detailed process, but they are producing open source software and working as a community. This handbook has a similar goal, to detail how an ideal InnerSource project would be, but there are not two organizations using the same InnerSource approach.
Thus, metrics useful for some contexts, for example technological organizations, might not apply to other context such as banks due to external factors such as legal regulations that are even different from country to country.
In addition to all of this, when measuring InnerSource, there are three main purposes to use metrics: check on going work, lead process improvement[^5] and motivational aspects.
Check on going work: this helps to understand where the development is right now. To be aware of the status helps to understand how fast things are changing when a new process is in the pipeline. This also helps to go from A to B and even trying several approaches to the same problem and have tests for this.
Lead process improvement: InnerSource means a change in how process works in the following. From a hierarchical way to a flatter way of working, InnerSource needs indicators to help to determine if that process improvement is properly working. And if this is not working, then using another approach or apply other policies.
Motivational aspects: InnerSource also means cultural change, and this is not usually taken into account in other methodologies. Indeed, this cultural change is identified in InnerSource as key. This should be the type of actions that will help to migrate from a traditional way of working to a more transparent and community- oriented way to work. And metrics can help to lead this process. First, to let developers know where they are and how their process is working, but also to have some fun within the work and competitions through challenges, hackathons and other measurable activities that lead to a more community-oriented organizations.
Depending on the InnerSource project, usage of the project could look something like:
Inclusion of a module in a build.
Calling and API.
Visiting a UI.
Metric
How it answers the question
Gotchas
We can see how many times an InnerSource project is reused. If there is extra information attached to that usage (like which business unit is the user) then we can see how widely across the company an InnerSource project is used.
When analyzing software development projects, there are several areas that can be analyzed. Traditional analysis are focused on code metrics, while the activity, community and process are key components to take into account. This handbook brings into context those areas not that well studied, but key for InnerSource: community and process.
InnerSource is basically all about direct interactions between developers, removing all of the hierarchical aspects remaining in the middle. It is all about creating communities and building social networks that allow the knowledge to fly around. InnerSource is about leaving developers to work with other developers (D2D) jumping over the traditional hierarchies. This helps to increase the development activity, break silos of information and allow innovation within the organization.
Activity: this component focuses on the basics. Any developer when interacting with the infrastructure of the organization leaves traces of activity. From commits to emails, from tickets activity to code review actions. Any of these traces is potentially measurable and many of these traces from a lot of developers display an organization chart with trends information. Measuring activity means quantifying the basic traces left by the developers. And when this information is aggregated and calculated over time, it is possible to obtain trends about this type of information. Usual activity measurements can be the number of commits up to some date, but also the increase or decrease of such number of commits when comparing to time frames. This can be extended to any potential event that may take place in a community of developers: emails, reviews, opening a ticket, closing a ticket, commenting on a code review or sending an email, but there are others more elaborated such as actions as mentor, newcomers activity, cross-referenced information like tickets referencing pull requests and those referencing commits, etc.
Community: people are key when talking about InnerSource. If one of the goals consists of flattering the hierarchical structure of an organization, we need to measure how people interact with other developers. Although this is again measured through the traces left by the developers, the analysis proposed are pretty focused on the people and how they build their own social networks. As each developer has her own interests, it is necessary to understand how developers are attracted and retained in a project. What policies are more successful than others when attracting those and who are leaders in the software development process. Usual measurements in this case are related to the analysis of the contributors funnel coming to an internal project, effectiveness of specific policies to attract developers (e.g., hackathons) and others more specialized such as territoriality (how territorial a developer is) or betweenness if we play with some indicators when building networks.
Process: thus we have a community of developers working together and leaving traces of their activity. But InnerSource is something more than spreading the knowledge across the organization. It is also a matter of reducing the time to market, increasing the velocity of development and building better products. That is the reason why process is also a needed area of analysis. Process can be defined as the bureaucracy followed by the developers to write a piece of code. In a more hierarchical structure, there might be some daily meetings where some decisions are taken and requirements implemented. That is process and if we want to improve that, we need to measure in first place key parameters of such process. In Inner- Source communities, process is quite similar to open source communities where code style, documentation, code review process, continuous integration, design meetings are part of this process. The main difference here is how developers directly interact with other developers, even being from different business units, even been far away geographically. That interaction is the one we need to polish and improve if we want to increase the quality and velocity of the development process. Indeed the ideal goal when improving process is that the total process takes exactly 0 seconds. Once a piece of code comes out of the developers minds, this works, and can be automatically deployed. How can we reach that situation? I do not know, but hey!, at least we can measure that and check how far we are from heaven.
Code: this aspect of the analysis is probably a well known one. There are usual metrics that determine the complexity of the code, modularity, test coverage, documentation coverage, and others. What we have when measuring this type of metrics is that we can control where our InnerSource method is leading those metrics. Increases in the code complexity or less test coverage may indicate unexpected behaviors that should be fixed and controlled.
Even when measuring and having data, people are still a key component of the strategy to deploy metrics and to have a driven-data process change. Metrics that lead to a process improvement or a motivational change are needed to be understood and approved by those that will be tracked. And there is not a need to measure everything, but to focus on specific areas of developers work that help to understand if this is working.
In addition to all of this, the metrics strategy is directly linked to the general purpose of the process update. From time to time organizations update their goals and those goals force a change in the metrics used to parse how far the organization is from those goals.
Thus, there are two main components in this process: goals and people. And developers must be aligned with the goals of the organization so everyone is walking the same path to success.
Developers and middle management should be part of the process to understand what the useful metrics are, to help to reject useless metrics and improve the metrics cycle. This will also help to introduce a data-driven development cycle and let the developers feel comfortable when using them.
When using metrics in InnerSource, we have already detailed the following purposes:
Awareness: be aware of the work in progress
Process Improvement: detect issues and understand the root causes of those issues (look for bottlenecks of your current process).
Motivational: let the development team follow some track (e.g.,: foster new contributions to a new project)
A usual way to apply a new methodology consists of following the PDCA (Plan-Do-Check-Act) to continuously improve. The steps ‘Plan’ and ‘Check’’ are the interesting ones for this metrics chapter, while the others are application of improvements in the process (‘Do’) and decision making (‘Act’).
The Planning actions consists of understanding the current situation of the process within the organization. For this and from a metrics point of view, it is necessary to retrieve qualitative and quantitative information. Quantitative in the sense of measuring what is taking place in the current development team and understand how developers and middle management use the infrastructure and interact among them. This will help to have a first glimpse of the situation of the project in a given moment in time.
From a qualitative point of view, feedback from developers and middle management is also necessary as they understand how the current process actually works. This is expected to fill the gap left by the quantitative approach and will help to understand if metrics are useful for their daily work, from a more managerial point of view or if they are useless at all for them.
Finally, it is necessary to define the goals, questions and metrics that will help to understand how much improvement we had when applying new policies in the software development process and other areas. Those metrics can also be linked to some actions such as alerts when the software process is not evolving as expected. And those alerts should end in actions or decisions in the ‘Act’ step.
Summarizing: qualitative and quantitative feedback, definition of goals and questions that will be checked later to measure success and tracking of the current situation of the project at the point of initiating this process.
Once the metrics and KPI’s were defined, the ‘Do’ step will take some weeks or months of work where developers work in a certain situation with some specific conditions. The ‘Check’ step will help to determine if this process was successful enough depending on the original goals defined at the ’Plan’ step. We may have improved the performance of the development model, but if the goal of this cycle was to attract more developers, we would have failed.
On top of this, the application of different policies or actions may take place in parallel and with different projects. Let’s take the example of selecting a new governance model for three different inner source projects: benevolent dictator, pure meritocracy where developers can work on their own and a mixing model of meritocracy but where developers are guided in the type of tasks they can work on.
This may help to follow a testing process where the governance model can be tested in small scale and the most successful govern later exported to others. This test A/B may help to understand the best governance model in certain circumstances. And having qualitative and quantitative feedback from the people involved in the process and the data sources they used provide great insights about the performance of the several models.
Software modeling and measurement: the goal question metric paradigm. Victor R. Basili
“Best Kept Secrets of Peer Code Review” by Jason Cohen, Steven Teleki, Eric Brown and others.
Chronological order:
Daniel Izquierdo Cortázar. Bitergia.
Chronological order:
Gregorio Robles. Universidad Rey Juan Carlos.
José Manrique López. Bitergia.
Kate Stewart. The Linux Foundation.