We Taught a 230M Language Model to Keep Learning on Android

Small language models can now run directly on phones. But most of them stop learning the moment they ship.

For personal AI, that feels like a strange stopping point. Some of the most useful signals arrive only after the model acts:

• Did the user dismiss the notification?
• Did they open it later?
• Did they rewrite the suggestion?
• Did they ask for it again?

These interactions contain useful information about the user, but they are delayed, private, and ambiguous. They are not clean labels, and they are not reliable scalar rewards.

To explore this problem, we built Online-SDFT, an open-source prototype that continually fine-tunes a small language model from delayed interactions while keeping the learning loop on the device.

The prototype uses:

• LiquidAI/LFM2.5-230M
• A rank-4 LoRA adapter
• ONNX Runtime Training
• A bounded on-device replay buffer
• An Android notification-routing testbed

Once the model has been provisioned, inference, interaction storage, replay, and adapter updates all happen locally.

Why standard fine-tuning is awkward here

Suppose the model receives a notification and chooses one of three actions:

1. Show it now
2. Save it for later
3. Archive it

Supervised fine-tuning would require a correct action for every notification. But the phone never observes what the ideal action was.

Reinforcement learning replaces the correct answer with a reward, but that reward is also difficult to define. Opening a notification does not necessarily mean it arrived at the right time. Ignoring it does not necessarily mean it was unimportant. The user may simply have been busy.

There is another complication: the model only observes the result of the action it actually took. If it archives a notification, it cannot know what would have happened had it shown the notification immediately.

What the phone receives is not a label or reward. It receives hindsight.

Using the same model as student and teacher

The core idea is simple: let the model reconsider its decision after seeing what happened.

At decision time, the student sees only the current context:

notification + time + local context

Later, the teacher sees the same context plus the observed outcome:

notification + time + local context + what the user did afterward

Because the teacher has more information, it can produce a better-informed distribution over the possible actions. We then distill that soft distribution into the student, which must make future decisions without access to the outcome.

Conceptually, the loop looks like this:

for interaction in stream:
    context = observe_context()

    action = student.sample(context)
    execute(action)

    hindsight = wait_for_outcome(action)

    with lora_disabled():
        target = model(context, hindsight)

    if causally_supported(action, hindsight):
        replay.add(context, target)
        update_lora(replay.sample_balanced())

There is no separate teacher model. The deployed model uses its LoRA adapter to act, while the same frozen base model reviews the completed interaction with the adapter disabled.

The teacher knows what happened. The student learns to anticipate what the teacher would conclude.

Why the target remains soft

A tempting alternative is to turn every outcome into a hard label.

For example:

dismissed notification → archive

But that inference is often too strong. A dismissal might mean the notification was irrelevant, badly timed, already understood from the preview, or simply interrupted by another task.

Online-SDFT therefore uses reliability-conditioned soft targets. Reliable outcomes can strongly support one action. Ambiguous outcomes redistribute probability only among the actions that remain plausible. Outcomes that reveal nothing useful produce no update.

This is important because it prevents the training loop from inventing counterfactual outcomes or treating every gesture as an explicit preference.

Two additions were important for online learning

Vanilla self-distillation does not specify how to collect useful interactions from a live, action-dependent stream.

Controlled exploration

A purely greedy model can lock into an early behavior and only collect evidence that confirms its own choices.

We add a small amount of exploration while the model is uncertain, then gradually taper it as confidence grows. This gives the system opportunities to observe outcomes from alternative actions without permanently making the serving policy random.

Small, balanced replay

Feedback is sparse and correlated. Several similar outcomes may arrive together, while rare but informative outcomes may occur only once.

Instead of updating from only the newest interaction, we retain a bounded window of recent lessons. Sampling balances feedback categories while favoring newer examples. The newest lesson is always included so that the model remains responsive to changes.

Replay is used only for training. It does not make the serving prompt longer.

Preliminary results

We evaluated six approaches on three paired synthetic notification streams, with 240 decisions per stream.

Here are selected results:

Online-SDFT matched the hidden sampled preference on 506 of 720 decisions.

The comparison with rejection fine-tuning was particularly interesting. Rejection fine-tuning used the same LoRA capacity but required a verified hard target. It accepted only 75 of 311 hindsight-teacher candidates. Online-SDFT could preserve graded information from outcomes that were useful but not strong enough to justify a one-hot label.

Replay also mattered substantially. Removing replay reduced preference accuracy from 70.3% to 39.6% and increased cumulative regret from 44.8 to 134.9.

These results are preliminary. There are only three synthetic streams, and the selected configuration was tuned on those same streams rather than confirmed on a separate held-out benchmark.

Running the loop on Android

The repository includes a separate Android project that runs the continual-learning loop on a physical device.

The frozen 230M-parameter base model makes notification-routing decisions. Once an outcome becomes available, ONNX Runtime Training updates the LoRA adapter. Adapter checkpoints and replay state remain in app-private storage and survive application restarts.

In one physical-phone test, repeatedly dismissing a notification teaches the model to keep it quiet. Asking for the notification again then changes the learned behavior so the next one is shown.

No server performs the update.

This is still an engineering prototype rather than a production notification manager. The current graph uses FP32 and targets high-memory ARM64 devices. Model export and initial provisioning still require a Linux host. We have also not yet completed systematic profiling of latency, peak memory, battery consumption, or thermal throttling.

Android notification-listener APIs also operate after a notification has been posted, so the current prototype demonstrates post-time routing rather than guaranteed suppression before an alert appears.

Try it

• Project and full write-up: Online-SDFT
• Source code and Android prototype: GitHub repository
• Runnable experiment: Google Colab notebook

The broader question we are interested in is:

What other applications could learn locally from delayed, action-dependent outcomes?

Notification routing is one example, but the same structure appears in writing suggestions, shortcut recommendations, application ranking, email assistance, and other forms of personal AI.

I would be especially interested in examples where the useful signal already exists on the device, but is too ambiguous to treat as a conventional label or reward.

Disclosure: I used AI assistance to help draft and edit this article. I reviewed the technical claims against the linked implementation and experiment artifacts before publication.