A Framework Supporting Replicable Performance Evaluations in Networking

View the Project on GitHub romain-jacob/triscale


A Framework Supporting Replicable Performance Evaluations in Networking

paper   code   tutorial   group

Following a live tutorial session? Here are the links you’re looking for

Part 1 Binder
Part 2 Binder
Notebook Basics Binder

Tell us about your experience!
If you followed one of our tutorials on TriScale, we would love to hear what you thought about it!
🙏    Please fill out this short survey (5min max)

👉    TriScale_survey

When designing their performance evaluations, networking researchers often encounter questions such as:

Despite the best intentions, researchers often answer these questions differently, thus impairing the replicability of evaluations and the confidence in the results.

Improving the standards of replicability has recently gained traction overall, as well as within the networking community. As an important piece of the puzzle, we developed a systematic methodology that streamlines the design and analysis of performance evaluations, and we have implemented this methodology into a framework called TriScale.

Reproduce our study

All data, code and information required to reproduce our study are openly available on GitHub.

We use Jupyter notebooks as wrappers for all our analyses. You can reproduce the paper results by running the notebooks locally or directly in your web-browser, thanks to the amazing service provided by!

See the file on GitHub for detailed instructions.


For more resources, check out the TriScale’s tutorial page.

Research replicability in embedded learning
ESWEEK Education, Virtual (October 2021)
video   slides

Tutorial: Supporting Replicable Networking Experiments with TriScale
ACM SIGCOMM 2021, Virtual (August 2021)
slides   video

Tutorial: Supporting Replicable Networking Experiments with TriScale
4th CPS-IoTBench Workshop, Virtual (May 2021)

IoTBench: Reproducibility challenge in wireless networking research
5th CROSS Symposium, Virtual (October 2020)

Confidence in experimental evaluations:
Time to do better than “Believe me, it’s true!”

EWSN Conference, Lyon, France (February 2020)

How to cite TriScale

  title         = {Designing Replicable Networking Experiments with TriScale},
  author        = {Jacob, Romain and Zimmerling, Marco and Boano, Carlo Alberto and Vanbever, Laurent and Thiele, Lothar},
  year          = 2021,
  doi           = {10.5070/SR31155408},
  year          = {2021},
  month         = nov,
  journal       = {JSys},
  volume        = {1},
  number        = {1},
  issn          = {2770-5501},
  url           = {},
  urldate       = {2021-11-15},
  area          = {Networking},
  artifacts_url = {},
  langid        = {english},
  review_url    = {},
Romain Jacob orcid   lead
Marco Zimmerling orcid
Carlo Alberto Boano orcid
Laurent Vanbever orcid
Lothar Thiele orcid

Contact us