Overview and Explanation of the AgileRanking™ Model
AgileRanking™ is a monthly ranking (#1 Feb. 2014 can be downloaded here) made on a selection of GitHub-hosted Open Source projects. It is built with 42 indicators that quantify the health of a project, defined with three analysis axis: Team Activity, Community Adoption and Code Quality. This post aims to provide readers with the model’s overview and explanations.
The AgileRanking™ model has been designed by Squoring Technologies consultants to assess and compare the global health of an open source project. Its aim is to provide – at a given moment or for a period of time – the information needed for decision taking, both for the End-Users (sustainability of the project, product and community quality…) and for the Development Team (risk identification, task prioritization, monitoring optimization, etc.).
Based on the « Goal Question Metrics » methodology (Victor Basili), the AgileRanking™ model revolves around three axis of analysis which are the main criteria when assessing and selecting an open source software solution:
- Team Activity: Is the project still alive? Is it sustainable? What is the reactiveness level provided by the community?
- Code Quality: Does the source code comply with best-practices? Can we easily maintain and extend it? How high is the density of the code Technical Debt?
- Community Adoption: Is the software really used? Is it appreciated and supported by the community? What level of audience and engagement on social networks?
As an open source application is intended to be used by a wide population (end-users, integrators, contributors, etc.), the perfect valuation and balancing of these 3 axis determines the chances of success of the project, at middle and long terms.
The AgileRanking™ Indicators
The three axis of the model (Team Activity, Code Quality and Community Adoption) are populated and valued with 42 indicators, as defined in the table below:
|Project Age||Team Activity||GitHub|
|Project Last Update||Team Activity||GitHub|
|Mailing List Recent Activity||Team Activity||GitHub|
|Closed Issue Ratio||Team Activity||GitHub|
|Closed Pull Ratio||Team Activity||GitHub|
|Maiing List Answer Rate||Team Activity||GitHub|
|Mailing List Answer Time||Team Activity||GitHub|
|Mailing List Comments||Team Activity||GitHub|
|Mailing List Subjects||Team Activity||GitHub|
|File Commits||Team Activity||GitHub|
|Download Ratio Trend||Community Adoption||GitHub|
|Stargazer Ratio Trend||Community Adoption||GitHub|
|Watcher Ratio Trend||Community Adoption||GitHub|
|Mailing List New Author Rate||Community Adoption||GitHub|
|Committer Ratio Trend||Community Adoption||GitHub|
|Contributor Ratio Trend||Community Adoption||GitHub|
|Mailing List Post Rate||Community Adoption||GitHub|
|Fork Ratio Trend||Community Adoption||GitHub|
|Open Pull Ratio Trend||Community Adoption||GitHub|
|Hashing Tweet Speed||Community Adoption|
|Mention Tweet Speed||Community Adoption|
|Stargazer Ratio Trend||Community Adoption||GitHub|
|Follower Ratio Trend||Community Adoption|
|Class Analysability Index||Code Quality||Squore|
|Function Analysability||Code Quality||Squore|
|Non-Compliant Analysability Rules||Code Quality||Squore|
|Adherence to Analysability Standards||Code Quality||Squore|
|Class Changeability Index||Code Quality||Squore|
|Function Changeability||Code Quality||Squore|
|Non-Compliant Changeability Rules||Code Quality||Squore|
|Adherence to Changeability Standards||Code Quality||Squore|
|Class Stability Index||Code Quality||Squore|
|Function Stability||Code Quality||Squore|
|Non-Compliant Stability Rules||Code Quality||Squore|
|Adherence to Stability Standards||Code Quality||Squore|
|Class Testability Index||Code Quality||Squore|
|Function Testability||Code Quality||Squore|
|Non-Compliant Testability Rules||Code Quality||Squore|
|Adherence to Testability Standards||Code Quality||Squore|
How to read the ranking
Each indicator has three visual elements, in order to provide a simple and intuitive reading:
- Color: the AgileRanking™ indicators are presented with a battery icon, with bars from 1 out of 5 (red, orange, yellow, light green, green). The more the battery is loaded and tends towards green, the better is the rank of the project indicator.
- Numeric Value: It represents the relative value of a project indicator versus all tracked projects in the ranking. This value ranges from 0 to 100.
- Trend: This is the evolution of an indicator compared with the previous version of the project. Indicators are monthly updated.
Model limitations & extension
This model is monthly fed with data that are automatically collected (Twitter and GitHub APIs, Squore code analyzers). It is not intended to be exhaustive and to provide « pure Agile » indicators. Depending on the availability of new data sources, the AgileRanking™ model would evolve to refine the valuation of the axis of analysis (i.e.: adding Velocity and Burndown indicators, if data from project sprints could be collected).
We remain at your disposal (agileranking [at] squoring.com) for any suggestion to improve the model and to demonstrate – in the context of your project – the analysis capabilities and power of the Squore solution.
Basically, Squore proposes a model dedicated to Agile developments, which is currently used by many industrial companies and IT organizations (providing continuous integration of data such as Business Value, Technical Debt, backlog information, etc.).