Track Awesome Msr Updates Weekly

A curated repository of software engineering repository mining data sets

🏠 Home · 🔍 Search · 🔥 Feed · 📮 Subscribe · ❤️ Sponsor · 😺 dspinellis/awesome-msr · ⭐ 346 · 🏷️ Computer Science

[ Daily / Weekly / Overview ]

Dec 28 - Jan 03, 2020

Tools

• astminer (⭐232) - Library and tool for mining of path-based representations of code and other data derived from ASTs.

• buckwheat (⭐23) - Multi-language tokenizer for extracting identifiers from source code.

• RefactoringMiner (⭐259) - Library/API for detection of refactorings in changes of Java code.

May 04 - May 10, 2020

Data Sets

• Eclipse AERI stacktraces - Collection of stacktraces of Exceptions encountered by users of the Eclipse IDE, as retrieved by the AERI reporting system.

• Software Heritage Graph Dataset - Graph of the development history and file metadata of >80 million software projects from various forges (GitHub, Gitlab, Debian, PyPI, Google Code, etc) in a deduplicated and unified representation (paper here).

Tools

• Perceval (⭐261) - Fetch repository data from tens of back-ends.

Research Outlets

• Outlets exclusively devoted to empirical software engineering research
  • Empirical Software Engineering journal
  • MSR: Mining Software Repositories conference
  • PROMISE: Predictive Models and Data Analytics in Software Engineering conference

Jul 08 - Jul 14, 2019

Data Sets

• Cryptocurrency GitHub Activity and Market Cap Dataset - Activity such as commits, stars, prices, and market cap of over 200 cryptocurrency projects on GitHub over time. Raw, historic data is also available.

• Linux Kernel 4.21 Call Graphs - The Linux Kernel 4.21 Call Graphs produced using CScout (⭐162).

Tools

• CryptOSS (⭐8) - Mine GitHub activity and market cap data for cryptocurrency projects.

Jul 01 - Jul 07, 2019

Data Sets

• Grammar Zoo - Collection of grammars of DSLs and GPLs, some extracted from metamodels and document schemata.

Jun 17 - Jun 23, 2019

Data Sets

• Unified Bug Dataset - Static source code based datasets which includes the Bugcatchers Bug Dataset, the Bug Prediction Dataset, the Eclipse Bug Dataset, the GitHub Bug Dataset, some datasets from the PROMISE repository.

Jun 03 - Jun 09, 2019

Data Sets

• GitHub Bug Dataset - Bug Dataset of 15 Java open-source projects characterized by static source code metrics.

May 13 - May 19, 2019

Tools

• Coming (⭐81) - A Java framework for analyzing code changes and mining instances of change patterns from Git repositories.

• DbDeo (⭐13) - Extract embedded SQL statements and detect database schema smells.

• Designite - Compute source code metrics and detect a variety of implementation, design, and architecture smells for C#.

• DesigniteJava (⭐138) - Compute source code metrics and detect a variety of implementation and design smells for Java.

• Puppeteer (⭐38) - Detect configuration smells in Puppet code.

Apr 22 - Apr 28, 2019

Research Outlets

• Outlets that publish empirical software engineering research
  • ACM Transactions on Software Engineering and Methodology (TOSEM)
  • ESEC/FSE: ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
  • ICSE: International Conference on Software Engineering
  • IEEE Software magazine
  • IEEE Transactions on Software Engineering
  • Journal of Systems and Software
  • SANER: IEEE International Conference on Software Analysis, Evolution and Reengineering

Mar 04 - Mar 10, 2019

Repositories

• SIR - Software-artifact infrastructure repository; Java, C, C++, and C# software together with test suites and fault data.

• PROMISE - About 20 datasets related to software engineering research.

• FLOSSmole - Collaborative collection and analysis of free/libre/open source project data.

• Zenodo - Software data collections in CERN's open-access repository.
  • Software Engineering Artifacts Can Really Assist Future Tasks
  • Empirical Software Engineering
  • Mining Software Repositories

Data Sets

• AndroidTimeMachine - Graph-based dataset of commit history of 8,431 real-world Android apps.

• AndroZoo - Collection of Android Applications.

• Bug Prediction Dataset - Collection of models and metrics from Eclipse JDT Core, PDE UI, Equinox Framework, Lucene, Mylyn, and their histories.

• Code Reviews - Code reviews of OpenStack, LibreOffice, AOSP, Qt, Eclipse.

• CoREBench - Collection of 70 realistically Complex Regression Errors that were systematically extracted from the repositories and bug reports of four open-source software projects: Make, Grep, Findutils, and Coreutils.

• Defects4J (⭐465) - Collection of 395 reproducible bugs collected with the goal of advancing software testing research.

• Enron Spreadsheets and Emails - All the spreadsheets and emails used in the paper 'Enron's Spreadsheets and Related Emails: A Dataset and Analysis'.

• Findbugs-maven (⭐1) - Set of FindBugs reports for the Java projects of the Maven repository.

• GHTorrent - Scalable, queriable, offline mirror of data offered through the GitHub REST API.

• GitHub on Google BigQuery - GitHub data accessible through Google's BigQuery platform.

• KaVE - Developer tool interaction data.

• Maven metrics (⭐0) - Collection of software complexity & sizing metrics for the Maven Repository.

• Maven Dependency Graph - Snapshot of the whole Maven Central taken on September 6, 2018, stored in a graph database.

• mzdata (⭐7) - Multi-extract and multi-level dataset of Mozilla issue tracking history.

• npm-miner (⭐0) - The dataset contains the analysis results of 5 open source software quality tools eslint, escomplex, nsp, jsinspect and sonarjs for 2000 popular (in terms of stars and downloads) npm packages.

• OCL Expressions on GitHub (⭐4) - Data set of 9188 OCL expressions originating from 504 EMF meta-models in 245 systematically selected GitHub repositories.

• RepoReapers Data Set - Data set containing a collection of engineered software projects from GHTorrent.

• STAMINA - (STAte Machine INference Approaches) data are used to benchmark techniques for learning deterministic finite state machines (FSMs).

• Stack Exchange - Anonymized dump of all user-contributed content on the Stack Exchange network.

• TravisTorrent - Provides free and easy-to-use Traivs CI build analyses.

• Ultimate Debian Database (UDD) - Data about various aspects of Debian (e.g. packages, bugs, mainteners) in the same SQL database.

• Unix history (⭐5.8k) - Git repository with 46 years of Unix history evolution.

Tools

• Boa - Domain-specific language and infrastructure that eases mining software repositories.

• ckjm - Chidamber and Kemerer Java Metrics.

• Diggit (⭐18) - Agile Ruby Tool to analyze Git repositories.

• GrimoireLab - Free/Libre/Open Source tools for Software Development Analytics.

• MetricMiner - Lean Java DSL to mine and extract data (e.g. commits, developers, modifications, diffs) from Git and SVN repositories.

• Maven-miner (⭐25) - Java tools and infrastructure to resolve the whole Maven dependency graph, hosted in Maven Central, in the form of a Neo4j Graph.

• PyDriller (⭐631) - Python Framework to analyse Git repositories.

• qmcalc (⭐54) - Calculate quality metrics from C source code.

• reaper (⭐99) - Python tool to compute a score for a repository from GHTorrent. The score quantifies the extent to which the project contained within the repository is engineered.

• VulData7 (⭐32) - Java framework enabling the automated collection of commits fixing vulnerabilities that are reported in NVD (links NVD with Git).