ArtifactsACSOS 2023

Accepted Artifacts

Call for Artifacts

ACSOS 2023 will offer an artifact track. Our aim is to encourage the community to build high-quality artifacts that support evaluating novel approaches, methods, or applications for autonomic and self-* systems that can also be reused, shared and extended by other researchers. Our long-term goal is to establish a collection of reusable evaluation setups and case studies. Creating and maintaining a catalog of artifacts will also support discussing, comparing and assessing the research conducted in this community.

Types of Artifacts

ACSOS definition of artifacts is inspired by the ACM definition, which defines an artifact as “a digital object that was either created by the authors to be used as part of the study or generated by the experiment itself. For example, artifacts can be software systems, scripts used to run experiments, input datasets, raw data collected in the experiment, or scripts used to analyze results.”

In particular, we are interested but not limited to:

• Software systems, including implementation testbeds or exemplars. We are looking for both simulated software systems and software deployable on real hardware, as well as mobile applications. These systems should highlight the specificities in the requirements, specifications, design, implementation, testing, and validation relevant to engineering self-* and autonomic systems or present challenges that these systems address.
• Frameworks that potentially include tools and services that illustrate and implement new approaches, techniques, or algorithms for self-* and autonomic systems.
• Data and datasets that include, e.g., logging data, system traces, sensor data, or raw survey data that can support various phases of the engineering process of self-* and autonomic systems.

This list is not exhaustive; authors with questions about the suitability of an artifact should reach out to the chairs. Additionally, the published artifact papers will be associated with IEEE Reproducibility Badges in the conference program and conference proceedings. We explain more about the badging and the required processes for preparing the artifact in the Submission tab.

Important Dates

Paper Submission Deadline: 	July 19th, 2023 (extension)
Notification to Authors: 	July 28th, 2023
Camera Ready Deadline: 	 	August 5th, 2023
ACSOS Conference:		September 25th - 29th, 2023

All times in Anywhere on Earth (AoE) timezone.

For more information on the artifacts submission to ACSOS 2023, please refer to the Submission tab.

Artifact Review Process and Selection Criteria

All submitted artifacts will be thoroughly evaluated by multiple reviewers of the artifact program committee. Each artifact will be evaluated according to the expectations set up 1) by the documentation of the artifact, and 2) by the artifact paper (in artifact standalone papers) or the full research paper (in artifact from a full research paper). In addition to just running the artifact, the reviewers may try to tweak provided inputs and create new ones, to test the limits of the system.

Artifacts will be evaluated according to the following criteria:

1. Understandability: Is the artifact easy to understand? Does the artifact identify and address a gap in the existing works and literature? Is the contribution of the artifact clearly explained?
2. Timeliness: Does the artifact address a problem that is timely? Is this value clearly explained in the paper?
3. Usability: Is the artifact executable: easy to download, install, configure and execute? Is the artifact operational: functional and usable? Can the artifact be reused by other researchers?
4. Quality of documentation: Is the artifact well-documented and structured? Is the readme of the artifact repository sufficient to run and execute the artifact? Does the documentation include the minimum system and environment requirements for usage (e.g., OS, CPU, RAM, GPU, Disk)? Are the limitations of the artifact identified and visibly documented?
5. Quality of code: Is the code of the artifact readable and well-commented?

The key benefit of investing the considerable work required to get an artifact published at ACSOS is the constant reuse of it by the community, which leads to continuously growing citations of the accompanying paper.

Camera Ready Submission for Companion Proceedings

STEP 1: Important Dates

• At least one author per paper must early pay the registration fee by August 5, 2023.
• Failure to register will result in your paper not being included in the proceedings.
• Final camera-ready manuscripts must be submitted by August 5, 2023.

STEP 2: Page Limits

6 pages including references for standalone papers.

Please note: Extra pages are not accepted.

STEP 3: Formatting Your Paper

• Submitted abstracts should not exceed 200 words.
• Final submissions to ACSOS 2023 must be formatted in US-LETTER page size, must use the two-column IEEE conference proceedings format, and must be prepared in PDF format. Microsoft Word and LaTeX templates are available at the IEEE “Author Submission Site” HERE. The templates are available on the left-hand-side tab “Formatting Your Paper”.
• Please, DO NOT include headers/footers or page numbers in the final submission.

STEP 4: Submitting Your Final Version

• Once the format of your paper has been verified and validated, you may submit your final version.
• All papers should be submitted using the submission system provided by IEEE “Author Submission Site” HERE.
• After you login to the IEEE “Author Submission Site”, please, follow the instructions as you click the “Next” button on the top right corner of the site. Please, enter the following information exactly as appeared on your paper:

  1. Paper ID (ART(Artifacts track) - Paper ID from EasyChair, eg. ART-4279)

  2. Names of authors, affiliations, countries, E-mail addresses,

  3. Titles, and abstracts.

• To submit your final manuscript click HERE.

STEP 5: Submitting a Signed Copyright Release Form

• ACSOS 2023 requires users to submit a fully digital version of the electronic IEEE Copyright-release Form (eCF). eCF is provided at the IEEE “Author Submission Site”.
• Follow the instructions in the IEEE “Author Submission Site” to properly fill-out, and submit the IEEE Copyright-release Form (eCF), including:
  1. Paper’s full title
  2. All authors names
  3. Conference title: 2023 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS)
  4. Signature (on appropriate line)

• The signed IEEE Copyright-release Form (eCF) should be submitted together with your camera-ready manuscripts on August 5th, 2023.

If you have any questions about the above procedures, please contact the Proceedings Chair Norha Villegas (nvillegas@icesi.edu.co)

Note: Extra pages are not allowed. Please complete each of the above steps - the conference organizers will not be responsible if your paper is omitted from the proceedings, is not available online on IEEE Xplore, or is subject to additional processing costs, if these steps are not performed.

Submission

SUBMISSION LINK: https://easychair.org/conferences/?conf=acsos2023

In ACSOS 2023 we call submissions of two artifacts:

1. artifact standalone paper and
2. artifact from a full research paper.

An artifact standalone paper comprises two parts: an artifact paper (max 6 pages, including references) and the artifact itself. The artifact paper should summarize the research problem, the research challenges in relation to system autonomy or the concrete self-* properties that the artifact addresses. The artifact papers have to adhere to IEEE formatting instructions.

Additionally, in the Artifact Track, we also aim to review and promote research artifacts from papers that were previously accepted as part of the Research Track. Namely, authors that have a full paper accepted as part of ACSOS 2023 can submit only their artifact for review within the Artifact Track.

The artifacts in both cases are expected to be prepared according to the IEEE Artifact Reproducibility Badges Policy and have a well-documented usage of the artifact. The badges will also be visible in the electronic versions of the papers in the IEEE Xplore document on the top-right of the page to denote the availability of the artifact. Please note that in the case of an artifact from a full research paper, the badge will be assigned to the research paper. Depending on the type of artifact, the papers will be assigned with either Code Reviewed and Dataset Reviewed badges.

Papers should be submitted via EasyChair. The papers should also contain a link from where the reviewers can access and download the artifact. Please note that at this stage, it does not need to be a link to the permanent repository. In case of artifact submissions from full research papers, we request the authors, upon submission of the artifact, to submit the pre-print or the submitted version of their accepted paper in which the artifact is used and, ideally, detailed and described in some of the sections of the paper. Artifacts must not have been previously published or concurrently submitted elsewhere.

The standalone artifact papers will be published in the conference proceedings companion. Additionally, the accepted artifacts from both the standalone artifact papers and published research papers will be given a chance to present their artifact at ACSOS 2023. Lastly, we will acknowledge the most useful artifact to the community with the best artifact award.

Preparing the Artifact for Submission

Authors must perform the following steps to submit an artifact:

Preparing the Artifact

While preparing the artifact, the authors need to prepare an installation package so the artifact can be installed and executed in the environment used by the reviewer. It is important that the documentation, e.g., the README.md in the artifact repository, provide sufficient information and instructions so that the reviewers with computer science background and reasonable knowledge of scripting and build tools can install, build, and run the artifact. When preparing the artifact, it is important to keep in mind: the accessibility of the artifact to other researchers and the fact that the artifact reviewers have very limited time to assess each artifact. Therefore, the setup for your artifact should take less than 30 minutes, or it is unlikely to be endorsed simply because the committee will not have sufficient time to evaluate it.

Suppose the artifact contains or requires the use of special tools or software. In that case, the authors must provide the artifact with a working environment in a virtual image (e.g., using Oracle Virtual Box or VMware) or a Docker container image. Therefore, the code should run on all three major platforms (Mac, Win, Linux) either natively or using Docker/VM. If designed for mobile applications, instructions should be clear and concise to assist reproducibility on the appropriate platform during the artifact review process. Alternatively, authors can provide a link to a web service that allows reviewers to execute the artifact (e.g., using Jupyter notebooks). In any case, the artifact should be exercisable and appropriately validated; otherwise, they may be desk-rejected.

Making the Artifact Available

Upon submission, the authors need to make sure that the artifact is publicly available for download under an open-source license so that the reviewers can access it. We suggest a link to a public repository, e.g., on GitHub under GPL. If the artifact contains large datasets or VM images, we recommend using open (commercial) archival repositories that guarantee long-time storage, e.g., figshare and Zenodo. We encourage the authors to use repositories dedicated to data sharing where no registration is necessary for those accessing the artifacts (e.g., please avoid using services such as GoogleDrive).
The artifacts accepted to the ACSOS 2023 must be made permanently available to the public by the time the camera-ready version of the paper is due. ACSOS 2023 reserves the right to archive artifacts to a central repository or ACSOS artifact webpage in the future.

Documenting the Artifact

The artifact should be complete and carefully documented. We summarize the requirements for documenting the artifact in the following.

The artifact must be self-contained, meaning it contains the artifact itself, which may include source code, executables, data, a virtual machine image, documents, and other content deemed relevant by the authors.

The artifact must include documentation that describes the artifact. Namely, the authors need to write and submit documentation explaining how to obtain the artifact package, how to unpack the artifact, how to get started, and how to use the artifacts in more detail. The entry point to the documentation should be easy to identify (e.g., README.md or index.html file in the top-level directory of the artifact). The documentation should contain:

• a Getting Started section in which the authors summarize the key elements of the artifact, which ideally enable the reviewers to run, execute or analyze the artifact without any technical difficulty. The requirements and the artifact limitations should be discussed. If applicable, descriptions of how the artifact can be customized and extended to be reused in different research contexts.
• step-by-step instructions on how to download, install, run, configure and execute the artifact, including how to check the results of the execution. These instructions should also show how the authors propose evaluating the artifact and how a new user can get started with the artifact. If applicable, descriptions of how the artifact can be customized and extended to be reused in different research contexts. And lastly, if the installation of the artifact includes multiple steps, then we expect the authors to provide scripts that automate the build and install process or to provide a virtual image or a Docker container image.
• where appropriate, descriptions of different sub-components of the artifacts, architectures, design diagrams, and figures shall be provided that assist the comprehension and evaluation of the artifact.
• a license file or statement describing the distribution rights. Note that a reusable artifact requires some kind of open-source license.

Optionally, the authors are encouraged to include in the documentation a link to a short video (YouTube, max. 5 minutes) demonstrating the artifact.