Repository navigation
Conversation
|
Hi @glaslos , just a gentle ping on this PR. Let me know if anything needs changing. Thanks! |
|
AI usage disclosure required? |
Yes, I used AI for code generation. However, I independently researched the problem, designed the architecture, identified the files and functions required, and determined the overall implementation approach. The code was generated with AI assistance, and I personally tested the implementation using both black-box and white-box testing. I'm also going through the issues you've highlighted in the review and working on the necessary changes. I'll push the updates to the existing PR shortly for your review. I'm happy to disclose my use of AI and take responsibility for the final contribution. |
|
Thanks for the review and suggestions! I have refactored the sensor statistics implementation to address all feedback. Summary of Changes
Verification & Tests
AI Assistance DisclosureIn accordance with CONTRIBUTING.md:
|
02946fd to
9fef003
Compare
|
Hi @glaslos ,
|
| for id, delta := range deltas { | ||
| if delta <= 0 { | ||
| continue | ||
| } | ||
| stat := entities.SensorStat{ | ||
| SensorID: id, | ||
| EventCount: delta, | ||
| LastEventAt: lastSeen[id], | ||
| } | ||
| if _, err := stmt.Exec(stat); err != nil { | ||
| return err | ||
| } | ||
| } |
There was a problem hiding this comment.
can you make this a single statment instead of a loop?
| // SetFlushInterval overrides the default periodic flush interval. | ||
| func (s *StatsService) SetFlushInterval(d time.Duration) { | ||
| s.flushInterval = d | ||
| } | ||
|
|
||
| // SetBroadcastInterval overrides the default broadcast interval. | ||
| func (s *StatsService) SetBroadcastInterval(d time.Duration) { | ||
| s.broadcastInterval = d | ||
| } |
There was a problem hiding this comment.
This is an anti pattern, pass the config to the constructor
|
|
||
| // Start loads initial stats from the database and launches periodic background tasks. | ||
| func (s *StatsService) Start(ctx context.Context) error { | ||
| if s.repo != nil { |
|
|
||
| // Stop stops background workers and commits any pending deltas to the database. | ||
| func (s *StatsService) Stop() error { | ||
| if s.cancel != nil { |
| if s.cancel != nil { | ||
| s.cancel() | ||
| } | ||
| s.wg.Wait() |
| } | ||
|
|
||
| // GetStatsForSensors retrieves current stats for the requested sensor IDs from memory. | ||
| // It matches by prefix when an event used a truncated 8-character ID or full UUID. |
| for { | ||
| select { | ||
| case msg := <-sub.msgs: | ||
| ctxTimeout, cancel := context.WithTimeout(ctx, 5*time.Second) |
| } | ||
|
|
||
| // PublishStats sends a message to all active WebSocket stats subscribers. | ||
| func (s *StatsService) PublishStats(msg []byte) bool { |
| case <-ctx.Done(): | ||
| return | ||
| case <-ticker.C: | ||
| _ = s.Flush() |
97a9cc0 to
64cd4c4
Compare
glaslos
left a comment
There was a problem hiding this comment.
Requesting changes, mainly for the unauthenticated stats WebSocket. Details inline.
| r.Post("/login", h.Login) | ||
| // Shared event links are unguessable IDs and must work without login. | ||
| r.Get("/events/{id}", h.GetEventByID) | ||
| r.Get("/sensors/stats/ws", h.StatsSubscribe) |
There was a problem hiding this comment.
This route is outside the auth group and broadcasts stats for every sensor of every user, including the full sensor UUIDs. Those UUIDs are what guards publish and /download, so anyone can harvest them. Please move it behind auth and filter the broadcast by owner.
| } | ||
| if count == 0 { | ||
| for totalID, c := range s.totals { | ||
| if (strings.HasPrefix(totalID, id) || strings.HasPrefix(id, totalID)) && c > 0 { |
There was a problem hiding this comment.
The prefix fallback lets another sensor's counts be attributed to the requested one. Publishing with an ID equal to the first 8 chars of a victim's UUID inflates their total. Match on the exact ID only.
…ynet#9) Add server-side sensor statistics using write-behind caching. Event counts are aggregated in memory and periodically flushed to SQLite in a single transaction, avoiding database writes per event on POST /api/publish. - Add StatsService managing in-memory sensor event totals, flusher, and WebSocket broadcaster lifecycle - Add sensor_stats table (sensor_id, event_count, last_event_at) with batch upsert in SensorRepo - Add GET /api/sensors/stats authenticated endpoint returning user's sensor counts directly from memory - Add GET /api/sensors/stats/ws WebSocket endpoint for real-time stats updates - Increment sensor stats on POST /api/publish using backend UTC timestamps - Update SensorList.svelte to display live event counts with WebSocket updates - Ensure non-negative counters and boundary protection against integer overflow - Add unit and integration tests for service, repo, handler, and server lifecycle Co-Authored-By: Google Antigravity <antigravity@google.com>
…t delete race - Move /api/sensors/stats/ws behind authentication and filter live broadcasts by sensor owner - Match sensor stats on exact ID only in GetStatsForSensors, removing prefix fallback - Add tombstones for deleted sensors and hold lock during SQLite batch flush to avoid re-insert races - Pass auth token parameter in SensorList.svelte when opening stats WebSocket - Add tests for auth rejection, broadcast isolation, exact matching, and tombstone protection Co-Authored-By: Google Antigravity <antigravity@google.com>
|
Hey @glaslos, moved /api/sensors/stats/ws behind authentication with owner-filtered broadcasts, switched to exact ID matching in GetStatsForSensors, and added tombstoning/locking during flush to eliminate the delete race. All frontend and backend tests pass; ready for review! |
64cd4c4 to
dc1a019
Compare
glaslos
left a comment
There was a problem hiding this comment.
There are still comments not addressed. Specifically in the stats service. I also want the Flush to not block the /api/publish endpoint
| r.Get("/", h.Index) | ||
| r.Get("/global.css", h.CSS) | ||
| r.Get("/myqueries", h.Index) | ||
| r.Get("/mysensors", h.Index) |
There was a problem hiding this comment.
This route is already registered
…ring flush - Remove duplicate /mysensors static route in backend/routes.go - Unblock /api/publish by executing SQLite flush outside s.mu lock - Pass StatsConfig to NewStatsService constructor instead of setter methods - Enforce non-nil repository on StatsService.Start - Bound StatsService.Stop with timeout to prevent shutdown hangs - Log background flush errors instead of silently discarding - Shorten WebSocket write timeout from 5s to 1s - Remove unused return on PublishStats - Batch upsert stats in a single multi-row SQL statement instead of a loop - Add unit test verifying IncrementStat is not blocked during SQLite flush Co-Authored-By: Google Antigravity <antigravity@google.com>
|
Hey @glaslos, Sorry, I missed a few things earlier. Anyway, I’ve addressed all the remaining feedback, everything is cleared now, and all tests are passing. The changes are ready for your review! |
Adds server-side sensor statistics using write-behind caching. Event counts are aggregated in memory and periodically flushed to SQLite in a single transaction, so there is no database write per event on POST /api/publish.
Closes #9
Changes:
How it works:
Sensor event -> POST /api/publish -> StatsCollector (RAM)
Testing:
Notes:
For a single-instance deployment, an in-memory map keeps things lightweight with no external dependencies. If the backend later scales horizontally, the map can be replaced with Redis (HINCRBY/INCRBY, plus Pub/Sub for WebSocket fan-out).