Jupyter Collaboration intermittently causes whole document to duplicate

[alasdair-macleod]

Description

This behaviour has been described before, but I am currently unable to find the issues that relate to it.
Collaboration appears to register the same user as more than one, duplicate the entire document after which validate(nb) updates all of the non unique cell_ids

The following (relatively old) sample logs are from an example with one user using one browser.
I have been trying for some time to find a more detailled explanation - but this appears to start when YDocExtension registers more users than are actually connected.

[I 2025-10-03 11:54:27.190 YDocExtension] Processed 29 Y patches in one minute | jupyter-cwjc51lxuvhnc05le5ww82
[I 2025-10-03 11:54:27.190 YDocExtension] Connected Y users: 2
...
...
validate(nb) jupyter-cwjc51lxuvhnc05le5ww82
/opt/conda/lib/python3.12/site-packages/nbformat/init.py:132: DuplicateCellId: Non-unique cell id 'e99af923' detected. Corrected to 'b12208b6'. jupyter-cwjc51lxuvhnc05le5ww82
validate(nb) jupyter-cwjc51lxuvhnc05le5ww82
/opt/conda/lib/python3.12/site-packages/nbformat/init.py:132: DuplicateCellId: Non-unique cell id 'a46d6d05' detected. Corrected to '5829dbf8'. jupyter-cwjc51lxuvhnc05le5ww82
validate(nb) jupyter-cwjc51lxuvhnc05le5ww82
/opt/conda/lib/python3.12/site-packages/nbformat/init.py:132: DuplicateCellId: Non-unique cell id 'be0e838c' detected. Corrected to '1c479935'. jupyter-cwjc51lxuvhnc05le5ww82
validate(nb) jupyter-cwjc51lxuvhnc05le5ww82
... (etc. until fininshed)

We run a cloud based service with tens of thousands of users. This issue is intermittent and hard to reproduce, but does happen to a significant number of our users.

Reproduce

I have found this issue very hard to reproduce.
Opening and closing the file repratedly has sometimes made it possible for me to replicate this.
The one common thread is that YDocExtension starts to register more users than are actually connected, despite being connected only once.
I have managed to replicate this running jupyter in a docker container on a single machine (i.e. not in a cloud environment)

Context

• Operating System and version: appears to be agnostic to operating system, various versions of Linux, MacOS and Windows tested
• Browser and version: again, browser agnostic.
• JupyterLab version: Jupyter Lab >= 4.4.3, Jupyter Collaboration >= 4.0

[krassowski]

@alasdair-macleod a lot of work was done to fix it, including very recently, in jupyter-ydoc v3.3.6 released five days ago:

• Deduplicate on YNotebook.get call jupyter-server/jupyter_ydoc#386
• Fix cell duplication when YStore diverges from disk jupyter-server/jupyter_ydoc#376

Can you confirm which version of jupyter-ydoc did you have installed? Later, can you confirm whether the upgrade to v3.3.6 helps?

[alasdair-macleod]

Hi, thanks - the first version we saw of this was 3.1.0. Most recently I've been testing 3.3.5.

Do you have a reliable way to reproduce the duplication behaviour? I don't find it easy to do and having confidence that I can reproduce it in an old version of jupyter-ydoc but not a new one would be a real game changer for me.

[krassowski]

Do you have a reliable way to reproduce the duplication behaviour?

The best I have is the reproduction instructions in jupyter-server/jupyter_ydoc#385 but they are not fully deterministic.

[alasdair-macleod]

Thanks a lot - I'll have a go with that and get back to you