AI-Security Challenge

Big challenge 01 · AI Security

Two frontiers of securing machine learning: AI models and federated learning.

Team registration openSubmissions open later

SEC

Category: Cybersecurity · Secure & Intelligent Autonomous Systems
Format: Day 2 hackathon · in person or online · on your own laptop
Tracks: 2 ranked · up to 2 per team
Teams: Solo or teams of up to 4
Submission deadline: 10 December 2026

Overview

Day 2's cybersecurity challenge has two tracks running side by side. Track 1 is about attacking AI models. Track 2 is about defending federated learning.

We gave each track the format that suits it: an attack benchmark for the first, a defence hackathon for the second. In both, most of your score comes from automatic grading on data and attacks you won't see until the end. That keeps the results objective, and it means an AI assistant on its own won't get you far.

There's no competition server. You work in Python on your own laptop, build or attack a real ML system, and get graded on a hidden test set. Knowing why your code works matters more than having it.

Tracks

A team may enter up to 2 of these. Each ranked track has its own leaderboard.

01Track 1

Attack benchmark · scored automatically

Red-Team the Model

You attack a deployed image classifier. Fool it, steal it, and get it to leak its training data, all within fixed budgets.

Scenario: A self-driving car relies on a model to read traffic signs. You play the attacker and show the three classic ways a deployed model fails: it can be fooled, it can be copied, and it can give away what it was trained on.
Target model: A ResNet-18 we trained on GTSRB (the German Traffic Sign Recognition Benchmark, 43 sign classes). It scores about 95% on clean images. There's also an optional DistilBERT text model. For Task A you get the full weights; for Task B you only get a local oracle with a query limit.
Threat model: Evasion (adversarial examples), model extraction (stealing) and membership inference (privacy). These are the standard ways a deployed classifier gets attacked.
Runs on: Python and PyTorch on a laptop. The model is small enough for a CPU, though a modest GPU will speed up Task A. Nothing runs live: you hand in files.

The tasks

A · Evasion (white-box)
Make the model misread signs on a hidden set of 200 images by adding perturbations within an L∞ budget of ε = 8/255. The attack is targeted, for example getting a STOP sign read as a speed limit. FGSM, PGD and Carlini–Wagner are the obvious places to start.
ScoredTargeted success rate. Any image that goes over the budget scores zero.
B · Extraction (black-box)
You get 10,000 queries to a local oracle. Use them to train a substitute model that behaves like the target. The point is to steal efficiently, not to brute-force it.
ScoredHow often your substitute agrees with the target on a hidden test set.
C · Membership inference (bonus)
You're given the model and a mixed pool of samples. Work out which ones it was trained on, using the fact that models tend to be over-confident on data they've already seen.
ScoredAUC of your membership scores.

Starter kit and how it runs

1Check your setup by reproducing the model's clean accuracy.
2Build each attack and score it yourself on the development set, staying inside the budgets.
3Hand in your files: adversarial images (.npy), the substitute model (.pt) and your membership scores (.csv).
4We run the scorer on the hidden set. Budgets are checked automatically.

Scoring

Task A: evasion success within budget45%
Task B: extraction agreement within the query budget35%
Task C: membership-inference AUC (bonus, capped)20%

You submit

Task A: adversarial images
A NumPy array of your perturbed images. Every image must stay within L∞ ε = 8/255.
NPY · up to 10 MB
Task B: substitute model
PyTorch weights for your substitute. If the file is over 10 MB, send a download link instead.
PT, PTH · up to 10 MB, or a link
Task C: membership scores (bonus)optional
CSV · up to 5 MB

02Track 2

Defence hackathon · auto-scored + jury

Byzantine Federation

You build an aggregation rule that keeps a shared medical model accurate and free of backdoors, even when some of the hospitals are malicious.

Scenario: Several hospitals train a diagnostic model together using federated learning. Patient images never leave the hospital; only model updates are shared. Some of those updates, though, come from compromised hospitals. Your job is to write the aggregation rule that keeps the shared model accurate and stops hidden backdoors getting in.
Model & data: A small CNN (2 conv layers, 2 dense layers) trained with FedAvg on PathMNIST, the 9-class pathology set from MedMNIST (28×28 colour images). 20 clients, non-IID split (Dirichlet α = 0.5), 50 rounds. It all runs in the Flower simulator on a laptop CPU.
Threat model: Up to 25% of clients can be malicious. Untargeted attacks: label flipping, sign flipping (Byzantine) and Gaussian noise. Targeted attack: a model-replacement backdoor, where a scaled-up update makes a trigger patch force a chosen class. Stealth attack: a norm-bounded backdoor designed to get past simple filters.
Runs on: Python, PyTorch and Flower on a laptop CPU. You submit an aggregation module and we run the final evaluation. Nothing runs live.

What teams build

Write a robust aggregation function, aggregate(client_updates) → global_update. You can add client scoring or filtering if you want. Feel free to start from a known method (Krum, Multi-Krum, trimmed mean, coordinate-wise median, Bulyan, FLTrust) and improve it, or design your own. The hard part is the trade-off: throw away too much and the model's accuracy on honest data drops with it.

Starter kit and how it runs

1Run the baselines first and watch plain aggregation fall apart under attack.
2Build a robust aggregator that holds up against the example attacks.
3Tune the balance between clean accuracy and robustness, then submit your module.
4We run it for 50 rounds against a hidden set of attacks, stealthy backdoor included.

Scoring

Clean accuracy with no attack (checks you haven't over-filtered)20%
Accuracy under the untargeted attacks30%
Backdoor resistance: how low you keep the trigger's success rate30%
Method and demo: soundness, originality, cost (jury)20%

You submit

Aggregation module
Your aggregate(client_updates) → global_update implementation, as a single .py file, or a .zip if it's split across modules.
PY, ZIP · up to 5 MB

How to submit

Register your team, then upload each deliverable from the participant portal. Your latest upload before the deadline is the one evaluated.

Track 1 · Red-Team the Model

Task A: adversarial images
A NumPy array of your perturbed images. Every image must stay within L∞ ε = 8/255.
NPY · up to 10 MB
Task B: substitute model
PyTorch weights for your substitute. If the file is over 10 MB, send a download link instead.
PT, PTH · up to 10 MB, or a link
Task C: membership scores (bonus)optional
CSV · up to 5 MB

Track 2 · Byzantine Federation

Aggregation module
Your aggregate(client_updates) → global_update implementation, as a single .py file, or a .zip if it's split across modules.
PY, ZIP · up to 5 MB

Every team

Slide deck for the jury
A short deck explaining your method and results.
PDF, PPTX · up to 10 MB

Shared rules, scoring and logistics

Teams and tracks

Teams of 3 to 4, open to students and professionals. Each team enters one track and can try the other for bonus points. As with the other Day 2 challenges, student-only teams get their own ranking.

How scoring works

Most of each track's score is measured automatically on a hidden test set. A jury covers what a script can't judge: how sound and original your method is, plus a short demo. Partial credit is built in, so any basic working solution earns points.

AI tools are allowed

We can't police AI use for teams joining online, so we don't try. Instead, the challenges are set up so that an assistant alone isn't enough: fixed budgets, hidden attacks, and problems that take several steps of reasoning to get right.

Infrastructure

Everything runs in Python and PyTorch on your own laptop. The models are small enough for a CPU; a modest GPU only makes a difference in Track 1. The only thing we run centrally is the final scoring on hidden data, so there's no competition server during the event. A simple shared scoreboard is optional.

What you submit

The files or module your track asks for, plus a short slide deck for the jury.

Difficulty: moderate

Each track comes with a starter notebook and working baseline code. You only need basic Python and machine learning, and partial credit means motivated students can finish with a score. The hidden tests will still separate the strongest teams.

Jury

A small panel of academics and engineers marks the method and demo.

Timeline

Key dates.

Team registration
Open now
Challenge day: both tracks run in parallel
17 December 2026
Results and awards
17 December 2026 · closing ceremony

Recognition

What’s at stake.

Prize for the first winner

Important

All datasets and tools mentioned are public and widely used in research and teaching: GTSRB (traffic signs), MedMNIST/PathMNIST (biomedical images) and the Flower federated learning framework. We generate all the attack data ourselves and don't use files from any other challenge.

Glossary

Adversarial example: An input altered by a tiny, often invisible, amount so that a model gets it wrong.
Model extraction: Rebuilding a copy of a model by querying it many times. In other words, stealing it without access to the weights.
Membership inference: Working out whether a particular record was in a model's training data. It's a privacy leak.
Federated learning: Training one shared model across many parties without moving their data. Only model updates are shared.
Backdoor: A hidden trigger planted during training that makes the model give an answer the attacker chose.
Robust aggregation: A way of combining model updates that holds up when a few of them are malicious (Krum or median, for example).
Hidden test set: Fresh data and attacks, kept secret until the end and used to grade every team the same way.

References

[1]GTSRB: German Traffic Sign Recognition Benchmark.benchmark.ini.rub.de/
[2]MedMNIST: standardised biomedical image datasets, including PathMNIST.medmnist.com/
[3]Flower: an easy-to-use federated learning framework.flower.ai/

AI-Security Challenge · ARC'26

Bring your best model.

Day 2's cybersecurity challenge has two tracks running side by side. Track 1 is about attacking AI models. Track 2 is about defending federated learning.

There's no competition server. You work in Python on your own laptop, build or attack a real ML system, and get graded on a hidden test set. Knowing why your code works matters more than having it.

01Track 1

Attack benchmark · scored automatically

Red-Team the Model

You attack a deployed image classifier. Fool it, steal it, and get it to leak its training data, all within fixed budgets.

Scenario: A self-driving car relies on a model to read traffic signs. You play the attacker and show the three classic ways a deployed model fails: it can be fooled, it can be copied, and it can give away what it was trained on.
Target model: A ResNet-18 we trained on GTSRB (the German Traffic Sign Recognition Benchmark, 43 sign classes). It scores about 95% on clean images. There's also an optional DistilBERT text model. For Task A you get the full weights; for Task B you only get a local oracle with a query limit.
Threat model: Evasion (adversarial examples), model extraction (stealing) and membership inference (privacy). These are the standard ways a deployed classifier gets attacked.
Runs on: Python and PyTorch on a laptop. The model is small enough for a CPU, though a modest GPU will speed up Task A. Nothing runs live: you hand in files.

The tasks

A · Evasion (white-box)
Make the model misread signs on a hidden set of 200 images by adding perturbations within an L∞ budget of ε = 8/255. The attack is targeted, for example getting a STOP sign read as a speed limit. FGSM, PGD and Carlini–Wagner are the obvious places to start.
ScoredTargeted success rate. Any image that goes over the budget scores zero.
B · Extraction (black-box)
You get 10,000 queries to a local oracle. Use them to train a substitute model that behaves like the target. The point is to steal efficiently, not to brute-force it.
ScoredHow often your substitute agrees with the target on a hidden test set.
C · Membership inference (bonus)
You're given the model and a mixed pool of samples. Work out which ones it was trained on, using the fact that models tend to be over-confident on data they've already seen.
ScoredAUC of your membership scores.

Starter kit and how it runs

1Check your setup by reproducing the model's clean accuracy.
2Build each attack and score it yourself on the development set, staying inside the budgets.
3Hand in your files: adversarial images (.npy), the substitute model (.pt) and your membership scores (.csv).
4We run the scorer on the hidden set. Budgets are checked automatically.

Scoring

Task A: evasion success within budget45%
Task B: extraction agreement within the query budget35%
Task C: membership-inference AUC (bonus, capped)20%

You submit

Task A: adversarial images
A NumPy array of your perturbed images. Every image must stay within L∞ ε = 8/255.
NPY · up to 10 MB
Task B: substitute model
PyTorch weights for your substitute. If the file is over 10 MB, send a download link instead.
PT, PTH · up to 10 MB, or a link
Task C: membership scores (bonus)optional
CSV · up to 5 MB

Teams and tracks

Teams of 3 to 4, open to students and professionals. Each team enters one track and can try the other for bonus points. As with the other Day 2 challenges, student-only teams get their own ranking.

Attack. Defend. Detect.

Choose your track.

Ship it from your team workspace.

One philosophy, three tracks.

Teams and tracks

How scoring works

AI tools are allowed

Infrastructure

What you submit

Difficulty: moderate

Jury

Key dates.

What’s at stake.

Bring your best model.

AI-Security Challenge

Attack. Defend. Detect.

Choose your track.

Ship it from your team workspace.

One philosophy, three tracks.

Teams and tracks

How scoring works

AI tools are allowed

Infrastructure

What you submit

Difficulty: moderate

Jury

Key dates.

What’s at stake.

Bring your best model.