Conversation
df280da to
00d279a
Compare
27fe54c to
ad099f8
Compare
9dbbd8c to
4ff0d67
Compare
0364555 to
b3122c7
Compare
|
In happy news Automerge, Autosurgeon and Samod have all gotten a release with the performance improvements. 🚀 🎉 I've upgraded them here. |
ec09d5b to
bb429fe
Compare
bb429fe to
04d5bb1
Compare
There was a problem hiding this comment.
Thanks Kaspar. There's a lot to process here. I've left some questions/comments focusing on the backend piece. I'll try to make another pass soon.
My overall impression is that I'm glad the complexity has shifted from Postgres queries/notification log to Rust code, because I think the Rust code will be easier to maintain and involves less magic. That said, the code footprint on the backend is quite large and going forward the complexity of adding new features to the backend will increase. So I'm really hoping that this is worth it on the frontend side, both in terms of dev experience and user experience.
34cb699 to
0aa7eda
Compare
|
Are the performance concerns mentioned here:
Addressed by this?:
I think benefits of the user-state-doc decrease if we need a parallel system for public docs. I'd be happy to help out with that or explore feasibility. |
7e14f8b to
dca18e9
Compare
dca18e9 to
ef29ea4
Compare
No, they are not.
It's not really public vs private it's just the number of docs I believe. We might need different ways of determining "these are the docs the user cares about right now", though with that we are sort of back to something akin to paginating I guess. |
6ef5e45 to
2919b19
Compare
2919b19 to
7aeba2f
Compare
* ENH: User state as an Automerge doc in backend * REFACTOR: Public lib for backend * ENH: Use Automerge UserState from frontend * REFACTOR: Move get_or_create user state function * REFACTOR: DbPermission * REFACTOR: Use same automerge doc creation as in test * ENH: Add parents to UserState DocInfo * ENH: Add children to DocInfo * ENH: Switch sidebar to use UserState * CLEANUP: Remove RefStub related code * CLEANUP: Remove unecessary refetch on delete doc * ENH: Update .sqlx queries * REFACTOR: Use fix in autosurgeon for Option<Text> * REFACTOR: Rename UserState UserSummary to UserInfo * ENH: Use String for user id in UserState * ENH: Sort side-bar sub-docs by createdAt * ENH: Show icons in documents and trash * ENH: Show "Analysis of" and "Diagram in" in document lists * ENH: Properly handle middle/Ctrl click for document lists * REFACTOR: DocumentList component * ENH: Add a link to Analysis of ... * REFACTOR: Use `A` from solid for document links * ENH: Show "Untitled" for empty doc title in list * ENH: Tweak user state doc init * TEST: Make sure doc is read in user state frontend test * REFACTOR: User state DB migration * TEST: Fix frontend user_state test isolation * ENH: Make user name changes notify user state docs * ENH: Add profile to UserState * ENH: Don't create user state in response to DB only * ENH: Handle profile updates more efficiently * ENH: Tweak reactivity around owners in document list * ENH: Tweak document list style and click behaviour * REFACTOR: Rename users field to known_users * ENH: Show docs with unknown parent as orphaned * ENH: Re-generate .sqlx * CLEANUP: Remove Paginated struct * TEST: Reduce wait times in user state tests * ENH: Switch to automerge/autosurgeon repo as crate source * ENH: Better types for type_name * TEST: Test user state url persistence * ENH: Persist user state urls in db * ENH: Clear persisted user states on restart * TEST: Split catlog and backend tests in CI * REFACTOR: ImmutabelString handling on user state permissions * REFACTOR: Add autosurgeon_datetime.rs * ENH: Handle unknown doc type in UserState DocInfo * ENH: Switch to depends_on and used_by in UserState * FIX: Move systemd "ready" to just before loop * DOC: Add comment about arbitrary DocInfo relations * ENH: Put user state doc id on users table * REFACTOR: Say "state doc ID" instead of "state doc URL" * ENH: Include public docs in UserState.documents * REFACTOR: Extract filterDocuments helper for document list filtering * REFACTOR: Extract reconcile_user_state helper for automerge reconciliation * ENH: Update user state from autosave listener instead of DB trigger Extract snapshot relations and update user state docs directly in the Rust autosave listener, eliminating the pg_notify round trip that re-read the snapshot from the database. * BUILD: Fix copying of api bindings for user_state * REFACTOR: Rename user_state_subscription migration * WIP: Use automerge batch-insertion branch * BUILD: Turn off debug-assertions for dev (automerge) * BUILD: Disable debug-assertions for rust deps * ENH: Bypass the DB for user state updates (#1151) * Revert "ENH: Include public docs in UserState.documents" This reverts commit c9eeb1c. * BUILD: Switch to release versions of automerge, autosurgeon and samod * TEST: Fix frontend test isolation by disallowing parallel runs * ENH: Simplify automerge user state updates (#1163) * DOC: Improve virtual_list doc comment * REFACTOR: Use isImmutableString from Automerge * REFACTOR: Define a DocumentType in notebook-types * REFACTOR: Move reconcile_into to a method on UserState * REFACTOR: Remove DbPermission intermediate type * CLEANUP: Remove profiling code from user_state.rs * FIX: UserState context needs to re-init on logout/login --------- Co-authored-by: Jason Moggridge <git@jmoggr.com>
This is ready for review but I suggest we wait for "Performance regression on main" (automerge/automerge#1159) to be resolved before deploying to production. As it stands it takes 100ms to create a new user state when the user first visits the site which manifest in a delay of seeing any documents if cold visiting
/documents.Closes the "server side state" part of #881 by implementing Automerge docs, derived from the db data, for every user state. Mutations are still handled through regular RPC calls. The
UserStatemainly holds the document lists and some user information and is put into aContexton the frontend.I couldn't resist slightly improving the UI of "My documents" and "Trash" since I had to replace them anyway. The overall feeling of reactivity of the document titles and the user names is very nice.
I am not sure how useful it is, but inspired by Benjamin Pierce's Colloquium talk, I made a commutative diagram of the paths I used property based testing on.
write_user_state_to_dbUserStateto the database (testing only).read_user_state_from_dbUserStatefrom the database.run_user_state_subscriptionuser_state_to_automergeUserStateinto an Automerge document.automerge_to_user_stateUserState(testing only).Note that
run_user_state_subscriptionwill be fine in normal usage but if you e.g. useALTER TABLEby loading a dump or something the automerge doc will be inconsistent with the DB.read_user_state_from_dbis run on request at the start so re-starting the server will fix the inconsistencies.Here is a very imperfect schema for the
UserState. (Imperfect since we can't really representVec<T>orOption<T>whereTis an entity in the schema on CatColab -- also papering over how things are referred to, directly or via ID.)https://catcolab.org/model/019cae54-2940-7971-9060-dcd2b8657ff5/analysis/019cae83-2b0a-7062-85a8-ceb4d1665c46