1. Preambles

This document may be revised to address omissions or inconsistencies, but it will not undergo substantial changes. Delegation Leaders are responsible for ensuring that all members of their delegation fully understand these rules and comply with them.

For additional documents regulating EJOI, please refer to:

• EJOI regulations
• EJOI syllabus
• Contest environment

2. Contest schedule
There are two contest days. On each day, contestants are given three tasks to be solved in 4 hours.

A 2-hour Practice Contest is held before the first contest day to familiarize all contestants with the grading system. The tasks used in the Practice Contest are published before the EJOI.

3. Grading system

Grading and evaluation take place on the grading system, which provides a similar execution environment for every submitted solution. The workstations have network access to the grading system.

Grading workstations will have similar hardware to contestant workstations. However, due to differences in software configurations, an identical execution environment between contestant and grading workstations is not guaranteed.

4. Tasks

Each contestant will receive the official English version of the tasks in an envelope on each contest day. The team leaders may translate the task statements for contestants, and the translated statements will be provided in the envelope with the English version. Each contestant will have online access to the official English version of the task statements and all translations in electronic format (PDF).

Each task is either a programming task (the solution is source code) or an output-only task (the solution is a set of output files). Each task is divided into a number of subtasks, each worth a portion of the total points.

For each programming task, time and memory limits are specified. In general, time and memory limits are generous (for example, double those required by the expected solution). The memory limit is on the overall memory usage, including executable code size, stack, dynamically allocated memory, etc.

For each task, the contestants can download a zip file from the grading system. For programming tasks, the file contains interface files, a sample grading program, an example implementation of the required source file, and the input and output files of the examples in the task statement. The provided example implementation demonstrates the interface of the task but does not solve the task. The sample grader provided on the workstation is not the same as the official grader used by the grading system.

5. Solutions and submissions

Contestants submit their solutions for the tasks via the grading system. Unless otherwise stated, the following restrictions apply to the submissions:

• programming tasks
  
  • Each submitted solution must be written in C++ and its size must not exceed 50 KiB.
  • Compilation of the program by the evaluation server must finish in at most 10 seconds and use at most 512 MiB of memory.
  • Submissions must not perform explicit input and output operations; instead, data must only be exchanged through the interfaces specified in the task statement. In particular, direct access to any file, including standard input or standard output, is forbidden (however, writing to standard error is allowed).
• output-only tasks
  
  • Each submitted solution must be a set of output files.
• Contestants must wait at least 30 seconds between submissions for a given task. This restriction does not apply in the last 15 minutes of the contest round.
• Contestants may perform at most 50 submissions for each task.

Using multiple threads is allowed. Note that the running time of the submission will be counted as a sum of running times of all threads. E.g., if there were two threads running for 5 seconds each (thus, the program finishes in 5 seconds), then the running time of the submission will be 10 seconds.

6. Scoring

Contestant’s points for each task are calculated as follows:

• Each subtask consists of some number of test cases.
• For each submission, the score for each test case is a fractional number from 0 to 1 inclusive, calculated based on the program execution and/or the output.
• The program execution on each test case is subject to time and memory limits, which are given in the grading system. If the program exceeds these limits, it receives a score of 0 for this test case. Usually, the score for a test case can only be 0 (the program failed the test case) or 1 (the program solved the test case).
• Unless otherwise stated, in each subtask, the submission is executed once per test case until some test case receives a score of 0, and then all following test cases are skipped and their score is 0.
• Some subtasks can have dependencies. The program will be executed on the test cases of a subtask that depends on previous ones only if all their test cases received scores greater than 0. If there is a test case in a subtask that receives a score of 0, all subtasks that are dependent on it will be skipped, and their test cases receive scores of 0.
• For each submission, the points for each subtask are calculated by multiplying the points for the subtask by the minimum of the scores for the test cases in the subtask, unless otherwise stated in the task statement.
• The final points for each subtask are the maximum of the points for this subtask across all submissions.
• The final points for each task are the sum of the points for its subtasks. This sum is rounded to 2 decimal places.

For example, consider a contestant who made two submissions on a task that contains two subtasks. If the first submitted solution got 30 points for the first subtask and 10 points for the second subtask, and the second solution got 0 points for the first subtask and 40 points for the second subtask, then the final points for this task will be 70.

The maximum points for each task are 100 points.

7. Feedback

Contestants can use the grading system to view the status of their submissions and get a short report on the compilation results of their source code.

For each submission, the grading system reports the points for each subtask. Unless otherwise stated, if a subtask is not fully solved, the grading system provides feedback only for the first test case among the lowest scored test cases in the subtask and all tests before that. The feedback includes the test case number and one of the following verdicts:

• OK – correct answer
• WA – wrong answer
• OK – partially correct answer (with a different color than the correct answer)
• TL – time limit exceeded
• ML – memory limit exceeded
• RE – runtime error (error during program execution, could be a memory limit problem)
• SK – skipped (the program is not evaluated on the test case)
• SE – system error (likely there is a problem with the task)

Unless otherwise stated, the “OK – partially correct answer” feedback is displayed when a submission receives a partial score for a test case in a subtask with partial scoring.

The test cases are ordered the same way in all the submissions. No information on the actual test cases, the output produced by the contestant solution, or any other execution details will be given to the contestant.

It should be noted that the scores and points reported in the feedback are only provisional. There are two ways in which they can change after they have been reported to the contestant:

• Due to a successful appeal after the contest.
• In some cases, the contestants’ submissions may be re-evaluated. This re-evaluation may sometimes lead to different total points (e.g., if a solution behaves non-deterministically or runs very close to the time or memory limit). In such cases, the final points for the submission are the points for its latest re-evaluation. This change in points cannot be appealed. Note that the final points for each subtask are still the maximum points over all submissions.

In the event of an error with the test data, the Scientific Committee will attempt to, but is not obligated to, follow the following process:

• Every attempt will be made to fix test data and re-evaluate all solutions as quickly as possible.
• Additional test data may be added only when the grading data significantly differs from the intention of the Scientific Committee from before the contest, e.g., the test data prepared before the contest was not properly uploaded to the grading system.
• Late detections of issues, especially during the last 2 hours of the contest, may be grounds for extending the length of the contest.

8. Quarantine

To protect task confidentiality, all direct and indirect contacts and communication between contestants and delegation leaders are prohibited from the moment tasks for a contest day are presented to the members of the GA until the end of that contest day. During this time, contestants must not communicate by any means, direct or indirect, with anyone who knows the tasks (except for the usual communication with the Scientific Committee during the contest).

Until the end of each contest day, GA meeting attendees must not share task-related information with anyone who does not know the contest tasks without explicit approval from the Scientific Committee. The contestants, the GA members, and anyone else who has had access to the tasks must obey any instructions which restrict their access to specific parts of the EJOI venue.

If a contestant violates the quarantine, the contestant may be subject to disqualification. If some other person associated with a national delegation violates the quarantine, then all contestants of that delegation may be subject to disqualification.

9. Supplies

9.1. Provided items

In the contest room, the following items will be provided:

• blank paper,
• writing tools,
• clarification Request Forms,
• snacks,
• water.

Any electronic or printed materials provided by the organizers during a contest round may be used by the contestants (e.g., a Users’ Guide to the Grading System, or any electronic documentation or reference manuals provided in the installed Contest Equipment or on the provided grading system).

9.2. Permitted items
On the contest days, contestants may bring the following items into the contest room:

• clothing,
• their ID badge (provided by the Organizing Committee),
• reasonable jewelry.

9.3. Permitted items with approval
Contestants may request to use additional items, provided that these items cannot transmit or store any data in electronic, printed, or other written format (other than their designed purpose). The following items are examples of items that may be requested:

• additional writing utensils,
• USB keyboards and mice (with no wireless communication and no programmable functions whose configuration is retained when unplugged),
• mouse pads,
• small mascots,
• English dictionaries,
• additional snacks,
• medicine and medical equipment,
• earplugs and earmuffs.

Bringing these additional items requires prior approval from the Technical Committee. A contestant must submit these items by leaving them in a designated container provided by the Technical Committee on their workstation during the Practice Contest.

As soon as the Practice Contest is over, the Technical Committee will check the submitted items. If there are rejected items, the delegation leader of the contestant will be notified, and they are allowed to resubmit replacements. The approved items will be kept by the Technical Committee and given to the contestants at the start of each contest day. However, during the contest, the Technical Committee may decide to remove any of these approved items if they deem that the item’s usage is disruptive to other contestants during the contest.

After the first contest day is over, a contestant must leave the submitted items in the same designated container provided by the Technical Committee on their workstation if they want to continue using these items on the second contest day. Contestants are allowed to submit new items or replacements during the time of analysis for the first contest day, to use them on the second contest day.

At the end of the Practice Contest and the first contest day, any unsubmitted items left on the contestants’ workstations will be cleared and not returned to the workstations. Hence, contestants should take all items with them at the end of the Practice Contest and each contest day.

At the end of the second contest day, contestants should take all the submitted items with them.

Regarding snacks, note that the Organizing Committee will provide all contestants with some number of snacks. In cases when a contestant would still like to bring in snacks, they should make sure that the snacks are not noisy or smelly, and are not disturbing for other contestants in any other way. In case of complaints from other contestants during the contest, the snack might be removed.

9.4. Prohibited items

Any attempts to bring any other items unlisted above into the contest rooms are considered cheating. In particular, the following items are strictly prohibited:

• any computing equipment (e.g., calculators, laptops, tablets),
• any keyboards or mice, with wireless communication or programmable functions, whose configuration is retained when unplugged,
• any books, manuals, written or printed materials,
• any data storage medium (e.g., CD-ROMs, USB drives, flash cards, micro-drives).
• any communication devices (e.g., mobile phones, radios of any sort, Bluetooth-enabled devices),
• watches of any type,
• any earphones, headphones, microphones, and speakers,
• backpacks and bags.

10. Contest

10.1. Starting the contest

All contestants must wear their ID badges during the contest. Each contestant will have a pre-assigned workstation. Contestants should be seated at least 5 minutes before the start of the contest.

Contestants must find their assigned workstation, sit down, and wait for the contest to begin without touching anything (such as keyboards, mice, pens, or paper).

10.2. Clarification requests

During the contest, contestants may ask questions regarding contest tasks, rules, and/or grading. Clarification Requests may be expressed either in English or the contestant’s preferred language. If required, delegation leaders will translate the Clarification Requests into English after they are submitted and before they are processed by the Scientific Committee.

The questions should be submitted using the grading system whenever possible. If this option is not available, either because typing in the contestant’s preferred language is not supported or due to technical issues, contestants can write the question on a printed Clarification Request Form.

Contestants will receive a reply from the Scientific Committee via the grading system, or in writing on the submitted Clarification Request Form.

Questions regarding the contest tasks should be phrased so that a yes/no answer will have a clear meaning. Contestants should not ask negative questions such as “Isn’t it true that…?” because the yes/no answer to such questions may cause confusion depending on the native language of the contestants. Instead, positive questions of the form “Is it true that…?” are recommended.

Questions regarding the contest tasks will be answered with one of the following (if the answer is via the grading system, it will be automatically translated in the selected language of the grading system):

• “Yes”
• “No”
• “No Comment/Answered in task description”
• “Invalid Question”– The question is most likely not phrased so that a yes/no answer would be meaningful. The contestant is encouraged to rephrase the question.

As a general rule, the Scientific Committee only answers “Yes” or “No”, when the corresponding part of the task statement is deemed incorrect, ambiguous, or potentially confusing.

The “No Comment/Answered in task description” answer is commonly given in the following situations:

• The answer to the question clearly follows from the task statement (either directly or indirectly).
• Determining the answer to the question is part of the contestant’s task.
• The answer to the question is not intended to be inferred from the task statement, and so the contestant should not assume either a positive or a negative answer. For example, when the task statement talks about a sequence, the contestant should not assume that the elements of the sequence are distinct, unless this is stated in the task, either directly or indirectly.

Additional elaboration of the answer may be provided if the Scientific Committee deems it necessary.

10.3. Assistance Requests

Requests not concerning contest tasks, rules, and/or grading would be considered as Assistance Requests. These Assistance Requests should be made by raising a colored card available on the contestant’s desk, depending on the type of request. For any other Assistance Requests, contestants should raise their hand to call the support staff for assistance.

The staff members will deliver Clarification Request Forms, help locate toilets and refreshments, and assist with computer and network problems. They will not answer questions about the contest tasks.

Contestants should not:

• Attempt to fix, debug, or even check for computer or network problems themselves; instead, they should ask for assistance and continue working on the tasks.
• Leave their seats until allowed to do so by the support staff.

10.4. Ending the contest

Three warnings will be given at 15 minutes, 5 minutes, and 1 minute before the end of the contest. Each warning will be given by an audible signal. The end of the contest will be announced both verbally and by an audible signal. At the announcement ending the contest, contestants must immediately stop working (unless they request extra time, please see the next paragraph) and wait quietly at their desks without touching the computers or anything on their desks. An additional announcement will be made instructing them to leave their tables and exit the contest room.

If a contestant thinks that they should be given extra time for the contest, they should send a Clarification Request either through the grading system or via a Clarification Request Form as early as possible. They should not leave their desk or talk to other contestants or team leaders after the contest ends and should continue to work. The Scientific Committee will then decide whether to award the extra time, and inform the contestant of the decision as early as possible. If some amount of extra time is granted, all submissions that are made during contest time + extra time would be evaluated.

However, note that the following issues will not be accepted as grounds for requesting extra time:

• Issues arising from the usage of programming tools, in particular IDEs (e.g., Code::Blocks, VS Code, and Eclipse) and debuggers. Contestants should be competent in the usage of the tools they decide to use. However, members of the Technical Committee may provide assistance.
• Issues that are resolved within 10 minutes, unless they happen less than 10 minutes before the end of the contest.
• The contestant loses a significant amount of time when trying to solve technical issues by themselves.

Instances where the contestant did not inform any support staff or make any Clarification request on the issue at all.

11. Cheating

Violating any of the following rules is considered cheating and may result in disqualification:

• Contestants must use only the workstation and account assigned to them on each competition day.
• Contestants must not try to tamper with or compromise the grading system.
• Contestants must not attempt to gain access to root or any account other than the one assigned to them.
• Contestants must not attempt to store information in any part of the file system other than the home directory for their account or the /tmp directory.
• Contestants must not touch any workstation other than the one assigned to them.
• Contestants must not attempt to access any machine on the network or the Internet, other than to access the grading system for usual purposes (e.g. submitting tasks, viewing submission results, downloading sample data, submitting Clarification Requests), and call for the support staff through the system; even running a single “ping” command is strictly prohibited and may lead to disqualification.
• Contestants must not attempt to reboot or alter the boot sequence of any workstation.
• Contestants must not communicate with other people during the competition, other than the support staff, and/or Scientific/Technical Committee members.
• Contestants must not reverse engineer the test data in order to solve the tasks in highly test-data-dependent manners. One example of such behavior is using the feedback system to extract the test data and then applying this knowledge to build solutions adapted to the specific test cases in the grading system. This behavior would be considered cheating only if a contestant submits a solution that would solve significantly fewer test cases correctly if the test data were replaced by an equivalent set of test cases (e.g., one generated with a different random seed).

12. Appeal process

The test cases used for grading will be made available in the grading system during the scheduled time for analysis after each competition. Contestants and team leaders may use the contestant’s workstations to verify that the points are given correctly.

A Team Leader may file an appeal by submitting an Appeal Form to the Scientific Committee at least 60 minutes before the final GA meeting of that competition day. The GA will be informed of where Appeal Forms can be collected, and where they can submit them to the Scientific Committee. All appeals are reviewed by the Scientific Committee, and decisions are communicated to the team leader. All appeals and their resolution will be summarized at the final GA meeting of that competition day.

If every submission of a task should be re-evaluated as a consequence of an accepted appeal, note that re-evaluation may result in a higher or lower points for any contestant. Should anyone’s points change after grading results have been published, new results will be published again. Points changes resulting from this are not appealable.