-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
Is your feature request related to a problem? Please describe.
ML researchers often run hundreds of experiments in Visdom. Currently, there is no structured way to categorize or filter these experiments, making it difficult to manage runs at scale.
Describe the solution you'd like
I propose an Experiment Tagging System designed for high reliability and performance:
-
Hybrid Metadata Storage:
Primary tags are persisted within each {env_id}.json for portability.
A lightweight tags_index.json is maintained on the server to allow OOM-safe "lazy-loading" of metadata without parsing large environment files during startup. -
Concurrency & Data Integrity:
Thread-safe writing using threading.Lock per environment.
Atomic filesystem updates using a write-to-temp + os.replace pattern to prevent file corruption during crashes or high-frequency updates. -
Real-time Sync & SDK Symmetry:
New WebSocket commands (tags_sync and tags_update) to broadcast changes to all connected frontends.
Symmetric Python SDK methods: vis.set_tags(tags, env, append) and vis.get_tags(env). -
UI Integration (Phase 2):
Tag-based filtering in the environment selector.
Tag management interface within the environment management modal.
Describe alternatives you've considered
Manual naming (prefixing) is currently the only way, but it's rigid and doesn't support multi-dimensional filtering.
Additional context
I am a GSoC 2026 applicant and have already implemented the Phase 1 (Backend & SDK) infrastructure. I will link the PR shortly.