Learn R Programming

reproducer

The R package reproducer is aimed to support reproducible research in software engineering. See the package homepage for details and examples.

Installation

One may install the stable version from CRAN:

install.packages('reproducer', dependencies = TRUE)

You can use devtools to install the development version from my web site:

install.packages("devtools", dependencies = T, repos = "https://cran.r-project.org/")
library(devtools)
devtools::install_url("https://madeyski.e-informatyka.pl/download/R/reproducer_0.7.0.tar.gz")
library(reproducer)

Motivation

The motivation is to support using robust statistical methods and reproducible research in software engineering via sharing data sets and code behind the published or just submitted papers.

Status

Citations

If you use reproducer, please cite it:

Lech Madeyski, Barbara Kitchenham, Tomasz Lewowski (2026). reproducer: Reproduce Statistical Analyses and Meta-Analyses. R package version 0.7.0. https://cran.r-project.org/package=reproducer

@Manual{reproducer,
title = {reproducer: Reproduce Statistical Analyses and Meta-Analyses},
author = {Lech Madeyski},
year = {2026},
note = {R package version 0.7.0},
url = {https://cran.r-project.org/package=reproducer} }

Lech Madeyski, Marian Jureczko (2015). Which process metrics can significantly improve defect prediction models? An empirical study. Software Quality Journal, vol. 23, no. 3, pp. 393-422 DOI: 10.1007/s11219-014-9241-7 Online: https://dx.doi.org/10.1007/s11219-014-9241-7

@Article{Madeyski15SQJ,
title = {Which process metrics can significantly improve defect prediction models? An empirical study},
author = {Lech Madeyski and Marian Jureczko},
journal = {Software Quality Journal},
year = {2015},
volume = {23},
number = {3},
pages = {393–422},
doi = {10.1007/s11219-014-9241-7},
url = {https://dx.doi.org/10.1007/s11219-014-9241-7} }

Marian Jureczko, Lech Madeyski (2015). Cross-project defect prediction with respect to code ownership model: An empirical study. e-Informatica Software Engineering Journal, vol. 9, no. 1, pp. 21-35 DOI: 10.5277/e-Inf150102 Online: https://dx.doi.org/10.5277/e-Inf150102

@Article{Jureczko15eInf,
title = {Cross-project defect prediction with respect to code ownership model: An empirical study},
author = {Marian Jureczko and Lech Madeyski},
journal = {e-Informatica Software Engineering Journal},
year = {2015},
volume = {9},
number = {1},
pages = {21–35},
doi = {10.5277/e-Inf150102},
url = {https://dx.doi.org/10.5277/e-Inf150102} }

Barbara A. Kitchenham, Lech Madeyski, David Budgen, Jacky Keung, Pearl Brereton, Stuart Charters, Shirley Gibbs and Amnart Pohthong (2017). Robust Statistical Methods for Empirical Software Engineering. Empirical Software Engineering, vol. 22, no.2, p. 579-630 DOI: 10.1007/s10664-016-9437-5 Online: https://dx.doi.org/10.1007/s10664-016-9437-5

@Article{Kitchenham17ESE,
title = {Robust Statistical Methods for Empirical Software Engineering},
author = {Barbara Kitchenham and Lech Madeyski and David Budgen and Jacky Keung and Pearl Brereton and Stuart Charters and Shirley Gibbs and Amnart Pohthong},
journal = {Empirical Software Engineering},
year = {2017},
volume = {22},
number = {2},
pages = {579–630},
doi = {10.1007/s10664-016-9437-5},
url = {https://dx.doi.org/10.1007/s10664-016-9437-5} }

Lech Madeyski and Barbara Kitchenham (2018) Effect Sizes and their Variance for AB/BA Crossover Design Studies. Empirical Software Engineering, vol. 23, no.4, p. 1982-2017 DOI: 10.1007/s10664-017-9574-5 Online: https://dx.doi.org/10.1007/s10664-017-9574-5

@Article{Madeyski18ESE,
title = {Effect Sizes and their Variance for AB/BA Crossover Design Studies},
author = {Lech Madeyski and Barbara Kitchenham},
journal = {Empirical Software Engineering},
year = {2018},
volume = {23},
number = {4},
pages = {1982–2017},
doi = {10.1007/s10664-017-9574-5},
url = {https://doi.org/10.1007/s10664-017-9574-5} }

Barbara Kitchenham, Lech Madeyski and Pearl Brereton (2020) Meta-analysis for families of experiments in software engineering: a systematic review and reproducibility and validity assessment. Empirical Software Engineering, vol. 25, no.1, p. 353-401 DOI: 10.1007/s10664-019-09747-0 Online: https://dx.doi.org/10.1007/s10664-019-09747-0

@Article{Kitchenham20ESE,
title = {Meta-analysis for families of experiments in software engineering: a systematic review and reproducibility and validity assessment},
author = {Barbara Kitchenham and Lech Madeyski and Pearl Pearl},
journal = {Empirical Software Engineering},
year = {2020},
volume = {25},
number = {1},
pages = {353–401},
doi = {10.1007/s10664-019-09747-0},
url = {https://doi.org/10.1007/s10664-019-09747-0} }

Tomasz Lewowski and Lech Madeyski (2020) Creating Evolving Project Data Sets in Software Engineering vol.851 of Studies in Computational Intelligence, p.1-14, Springer DOI: 10.1007/s10664-019-09747-0 Online: https://dx.doi.org/10.1007/s10664-019-09747-0

@InBook{Lewowski20SCI,
title = {Creating Evolving Project Data Sets in Software Engineering},
booktitle = {Integrating Research and Practice in Software Engineering},
chapter = {Creating Evolving Project Data Sets in Software Engineering},
author = {Tomasz Lewowski and Lech Madeyski},
editor = {Stanislaw Jarzabek and Aneta Poniszewska-Mara{’{n}}da and Lech Madeyski},
year = {2020},
volume = {851},
series = {Studies in Computational Intelligence},
pages = {1–14},
publisher = {Springer},
doi = {10.1007/978-3-030-26574-8_1},
url = {https://doi.org/10.1007/978-3-030-26574-8_1} }

Barbara Kitchenham, Lech Madeyski, Giuseppe Scanniello, and Carmine Gravino (2022) The importance of the correlation in crossover experiments IEEE Transactions on Software Engineering, vol. 48, no.8, p.2802-2813 DOI: 10.1109/TSE.2021.3070480 Online: https://doi.org/10.1109/TSE.2021.3070480

@Article{Kitchenham22TSE,
title = {The importance of the correlation in crossover experiments},
author = {Barbara Kitchenham and Lech Madeyski and Giuseppe Scanniello and Carmine Gravino},
journal = {IEEE Transactions on Software Engineering},
year = {2022},
volume = {48},
number = {8},
pages = {2802–2813},
doi = {10.1109/TSE.2021.3070480},
url = {https://doi.org/10.1109/TSE.2021.3070480} }

Copy Link

Version

Install

install.packages('reproducer')

Monthly Downloads

403

Version

0.7.0

License

GPL (>= 2)

Maintainer

Lech Madeyski

Last Published

July 12th, 2026

Functions in reproducer (0.7.0)

KitchenhamMadeyskiBrereton.ABBAMetaAnalysisReportedResults

KitchenhamMadeyskiBrereton.ABBAMetaAnalysisReportedResults data
KitchenhamEtAl.CorrelationsAmongParticipants.Torchiano17JVLC

KitchenhamEtAl.CorrelationsAmongParticipants.Torchiano17JVLC data
KitchenhamEtAl.CorrelationsAmongParticipants.Scanniello14TOSEM

KitchenhamEtAl.CorrelationsAmongParticipants.Scanniello14TOSEM data
KitchenhamEtAl.CorrelationsAmongParticipants.Ricca14TOSEM

KitchenhamEtAl.CorrelationsAmongParticipants.Ricca14TOSEM data
KitchenhamEtAl.CorrelationsAmongParticipants.Scanniello14JVLC

KitchenhamEtAl.CorrelationsAmongParticipants.Scanniello14JVLC data
LaplaceDist

LaplaceDist
KitchenhamMadeyskiBudgen16.FINNISH

KitchenhamMadeyskiBudgen16.FINNISH data
KitchenhamMadeyskiBudgen16.PolishData

KitchenhamMadeyskiBudgen16.PolishData data
KitchenhamMadeyskiBrereton.MetaAnalysisReportedResults

KitchenhamMadeyskiBrereton.MetaAnalysisReportedResults data
Madeyski15EISEJ.OpenProjects

Madeyski15EISEJ.OpenProjects data
KitchenhamMadeyskiBrereton.DocData

KitchenhamMadeyskiBrereton.DocData data
KitchenhamMadeyskiBrereton.ExpData

KitchenhamMadeyskiBrereton.ExpData data
KitchenhamMadeyskiBrereton.ReportedEffectSizes

KitchenhamMadeyskiBrereton.ReportedEffectSizes data
KitchenhamMadeyskiBrereton.ABBAReportedEffectSizes

KitchenhamMadeyskiBrereton.ABBAReportedEffectSizes data
KitchenhamMadeyskiBudgen16.PolishSubjects

KitchenhamMadeyskiBudgen16.PolishSubjects data
KitchenhamMadeyskiBudgen16.SubjectData

KitchenhamMadeyskiBudgen16.SubjectData
Madeyski15SQJ.NDC

Madeyski15SQJ.NDC data
MadeyskiKitchenham.MetaAnalysis.PBRvsCBRorAR

MadeyskiKitchenham.MetaAnalysis.PBRvsCBRorAR data
MadeyskiKitchenham.EUBASdata

MadeyskiKitchenham.EUBASdata data
KitchenhamMadeyski.SimulatedCrossoverDataSets

KitchenhamMadeyski.SimulatedCrossoverDataSets data
Madeyski15EISEJ.PropProjects

Madeyski15EISEJ.PropProjects data
KitchenhamEtAl.CorrelationsAmongParticipants.Scanniello17TOSEM

KitchenhamEtAl.CorrelationsAmongParticipants.Scanniello17TOSEM data
KitchenhamMadeyskiBudgen16.DiffInDiffData

KitchenhamMadeyskiBudgen16.DiffInDiffData data
KitchenhamEtAl.CorrelationsAmongParticipants.Scanniello15EMSE

KitchenhamEtAl.CorrelationsAmongParticipants.Scanniello15EMSE data
Madeyski15EISEJ.StudProjects

Madeyski15EISEJ.StudProjects data
MadeyskiLewowski.IndustryRelevantGitHubJavaProjects20190324

MadeyskiLewowski.IndustryRelevantGitHubJavaProjects20190324 data
KitchenhamMadeyskiBudgen16.COCOMO

KitchenhamMadeyskiBudgen16.COCOMO data
NP2GMetaAnalysisSimulation

NP2GMetaAnalysisSimulation
MetaAnalysisSimulations

MetaAnalysisSimulations
MadeyskiLewowski.IndustryRelevantGitHubJavaProjects20191022

MadeyskiLewowski.IndustryRelevantGitHubJavaProjects20191022 data
MunzelBrunner02.PGI

Patient Global Impression (PGI) data from Munzel and Brunner (2002)
NP4GMetaAnalysisSimulation

NP4GMetaAnalysisSimulation
PrepareForMetaAnalysisGtoR

PrepareForMetaAnalysisGtoR
PHatonesidedTestStatistics

PHatonesidedTestStatistics
RandomizedBlocksAnalysis

RandomizedBlocksAnalysis
RandomExperimentSimulations

RandomExperimentSimulations
PHattwosidedTestStatistics

PHattwosidedTestStatistics
RandomizedBlocksExperimentSimulations

title RandomizedBlocksExperimentSimulations description This function performs multiple simulations of 4 group balanced randomised Block experiments with two control groups and two treatment groups where one control group and one treatment group are assigned to block 1 and the other control group and treatment group are assigned to block 2. The simulations are based on one of four distributions and a specific group size. The function identifies the average value of the non-parametric effect sizes P-hat, Cliff' d and their variances and whether ot not the statistics were significant at the 0.05 level. We also present the values of the t-test as a comparison.
RandomizedBlockDesignEffectSizes

RandomizedBlockDesignEffectSizes
PHat.test

PHat.test
boxplotAndDensityCurveOnHistogram

boxplotAndDensityCurveOnHistogram
RandomizedDesignEffectSizes

RandomizedDesignEffectSizes
calcEffectSizeConfidenceIntervals

calcEffectSizeConfidenceIntervals
calc.a

calc.a
aggregateIndividualDocumentStatistics

aggregateIndividualDocumentStatistics
boxplotHV

boxplotHV
calcCliffdTestStatistics

calcCliffdTestStatistics
calcPHatConfidenceIntervals

calcPHatConfidenceIntervals
calc.b

calc.b
calcCliffdConfidenceIntervals

calcCliffdConfidenceIntervals
calculate4GBias

calculate4GBias
calculate2GType1Error

calculate2GType1Error
calculateKendalltaupb

@title calculateKendalltaupb @description Computes point bi-serial version of Kendall's tau plus a 1-alpha confidence interval using the method recommended by Long and Cliff (1997). The algorithm is based on Wilcox's code but was extended to return the consistent variance and the confidence intervals based on the t-distribution. Also added a Diagnostic parameter to output internal calculations.
calculate2GBias

calculate2GBias
calculateGroupSummaryStatistics

calculateGroupSummaryStatistics
calculateHg

calculateHg
calcPHatMATestStatistics

calcPHatMATestStatistics
calculateCliffd

calculateCliffd
calculateBasicStatistics

calculateBasicStatistics
calculateLargeSampleRandomizedBlockDesignEffectSizes

calculateLargeSampleRandomizedBlockDesignEffectSizes
compute_mcc

compute_mcc
compute_wmcc

compute_wmcc
calculateSmallSampleSizeAdjustment

calculateSmallSampleSizeAdjustment
checkIfValidDummyVariable

checkIfValidDummyVariable
getEffectSizesABBAIgnoringPeriodEffect

getEffectSizesABBAIgnoringPeriodEffect
getEffectSizesABBA

getEffectSizesABBA
calculateNullESAccuracy

calculateNullESAccuracy
calculateMAType1Error

calculateMAType1Error
densityCurveOnHistogram

densityCurveOnHistogram
doLM

doLM
fmt

fmt
metaanalyse.Cliffd

metaanalyse.Cliffd
calculate4GType1Error

calculate4GType1Error
calculateLargeSampleRandomizedDesignEffectSizes

calculateLargeSampleRandomizedDesignEffectSizes
llm4screenlit_metrics

llm4screenlit_metrics
effectSizeCI

effectSizeCI
calculateMABias

calculateMABias
printXTable

printXTable
proportionOfSignificantTValuesUsingCorrectAnalysis

proportionOfSignificantTValuesUsingCorrectAnalysis
constructEffectSizes

constructEffectSizes
calculatePopulationStatistics

calculatePopulationStatistics
calculatePhat

calculatePhat
reproduceTableWithEffectSizesBasedOnMeanDifferences

reproduceTableWithEffectSizesBasedOnMeanDifferences()
getSimulationData

getSimulationData
crossoverResidualAnalysis

crossoverResidualAnalysis
reproduceTableWithPossibleModeratingFactors

reproduceTableWithPossibleModeratingFactors()
simulate2GExperimentData

simulate2GExperimentData
searchForIndustryRelevantGitHubProjects

searchForIndustryRelevantGitHubProjects
proportionOfSignificantTValuesUsingIncorrectAnalysis

proportionOfSignificantTValuesUsingIncorrectAnalysis
reproduceMixedEffectsForestPlotWithExperimentalDesignModerator

reproduceMixedEffectsForestPlotWithExperimentalDesignModerator()
reproduceSimulationResultsBasedOn500Reps1000Obs

reproduceSimulationResultsBasedOn500Reps1000Obs
pairedRankTest

pairedRankTest
readExcelSheet

readExcelSheet
getTheoreticalEffectSizeVariancesABBA

getTheoreticalEffectSizeVariancesABBA
wmcc_sensitivity

wmcc_sensitivity
reproduceTableWithSourceDataByCiolkowski

reproduceTableWithSourceDataByCiolkowski
percentageInaccuracyOfLargeSampleVarianceApproximation

percentageInaccuracyOfLargeSampleVarianceApproximation
reproduceForestPlotRandomEffects

reproduceForestPlotRandomEffects()
plotOutcomesForIndividualsInEachSequenceGroup

plotOutcomesForIndividualsInEachSequenceGroup
metaanalyse.PHat

metaanalyse.PHat
metaanalyseSmallSampleSizeExperiments

metaanalyseSmallSampleSizeExperiments
reproduceTablesOfPaperMetaAnalysisForFamiliesOfExperiments

reproduceTablesOfPaperMetaAnalysisForFamiliesOfExperiments
transformZrtoHg

transformZrtoHg
transformZrtoHgapprox

transformZrtoHgapprox
pairedSignTest

pairedSignTest
transformHgtoR

transformHgtoR
transformHgtoZr

transformHgtoZr
reproduceMixedEffectsAnalysisWithExperimentalDesignModerator

reproduceMixedEffectsAnalysisWithExperimentalDesignModerator()
reproduceMixedEffectsAnalysisWithEstimatedVarianceAndExperimentalDesignModerator

reproduceMixedEffectsAnalysisWithEstimatedVarianceAndExperimentalDesignModerator()
transformRtoZr

transformRtoZr
simulate4GExperimentData

simulate4GExperimentData
simulateRandomizedBlockDesignEffectSizes

simulateRandomizedBlockDesignEffectSizes
transformRtoHg

transformRtoHg
rSimulations

rSimulations
testfunctionParameterChecks

testfunctionParameterChecks
transformZrtoR

transformZrtoR
varStandardizedEffectSize

varStandardizedEffectSize
simulateRandomizedDesignEffectSizes

simulateRandomizedDesignEffectSizes
AnalyseResiduals

AnalyseResiduals
CalculateRLevel1

CalculateRLevel1
Cliffd.test

Cliffd.test
CatchError

CatchError
Calc4GroupNPStats

Calc4GroupNPStats
ExtractExperimentData

ExtractExperimentData
ExtractGroupSizeData

ExtractGroupSizeData
ConstructLevel1ExperimentRData

ConstructLevel1ExperimentRData
Ciolkowski09ESEM.MetaAnalysis.PBRvsCBRorAR

Ciolkowski09ESEM.MetaAnalysis.PBRvsCBRorAR data
CalculateLevel2ExperimentRData

CalculateLevel2ExperimentRData
ExtractMAStatistics

ExtractMAStatistics
KitchenhamEtAl.CorrelationsAmongParticipants.Abrahao13TSE

KitchenhamEtAl.CorrelationsAmongParticipants.Abrahao13TSE data
KitchenhamEtAl.CorrelationsAmongParticipants.Gravino15JVLC

KitchenhamEtAl.CorrelationsAmongParticipants.Gravino15JVLC data
KitchenhamEtAl.CorrelationsAmongParticipants.Reggio15SSM

KitchenhamEtAl.CorrelationsAmongParticipants.Reggio15SSM data
KitchenhamEtAl.CorrelationsAmongParticipants.Madeyski10

KitchenhamEtAl.CorrelationsAmongParticipants.Madeyski10 data
KitchenhamEtAl.CorrelationsAmongParticipants.Ricca10TSE

KitchenhamEtAl.CorrelationsAmongParticipants.Ricca10TSE data
KitchenhamEtAl.CorrelationsAmongParticipants.Romano18ESEM

KitchenhamEtAl.CorrelationsAmongParticipants.Romano18ESEM data
KitchenhamEtAl.CorrelationsAmongParticipants.Scanniello14EASE

KitchenhamEtAl.CorrelationsAmongParticipants.Scanniello14EASE data
ExtractSummaryStatisticsRandomizedExp

ExtractSummaryStatisticsRandomizedExp