Commit Graph

48 Commits

Author SHA1 Message Date
Morgan Pretty a41f1c1366 Fixed the broken tests
Cleaned up the Dependencies so that tests can run synchronously without having to custom set queues as much
Sorted out the crypto and network dependencies to avoid needing weird dependency inheritance
Fixed the flaky tests so they are no longer flaky
Fixed some unexpected JobRunner behaviours
Updated the CI config to use a local build directory for derivedData (now works with build tweaks)
2023-08-01 14:27:41 +10:00
Morgan Pretty e768bebe6d Merge remote-tracking branch 'upstream/dev' into feature/job-runner-unit-tests
# Conflicts:
#	Session.xcodeproj/project.pbxproj
#	Session/Meta/Session-Prefix.pch
#	Session/Notifications/SyncPushTokensJob.swift
#	Session/Utilities/BackgroundPoller.swift
#	SessionMessagingKit/Configuration.swift
#	SessionMessagingKit/Database/Models/Profile.swift
#	SessionMessagingKit/Jobs/Types/AttachmentDownloadJob.swift
#	SessionMessagingKit/Jobs/Types/AttachmentUploadJob.swift
#	SessionMessagingKit/Jobs/Types/DisappearingMessagesJob.swift
#	SessionMessagingKit/Jobs/Types/FailedAttachmentDownloadsJob.swift
#	SessionMessagingKit/Jobs/Types/FailedMessageSendsJob.swift
#	SessionMessagingKit/Jobs/Types/GroupLeavingJob.swift
#	SessionMessagingKit/Jobs/Types/MessageReceiveJob.swift
#	SessionMessagingKit/Jobs/Types/MessageSendJob.swift
#	SessionMessagingKit/Jobs/Types/NotifyPushServerJob.swift
#	SessionMessagingKit/Jobs/Types/RetrieveDefaultOpenGroupRoomsJob.swift
#	SessionMessagingKit/Jobs/Types/SendReadReceiptsJob.swift
#	SessionMessagingKit/Jobs/Types/UpdateProfilePictureJob.swift
#	SessionMessagingKit/Sending & Receiving/MessageSender.swift
#	SessionMessagingKit/Sending & Receiving/Pollers/ClosedGroupPoller.swift
#	SessionMessagingKit/Utilities/AppReadiness.m
#	SessionMessagingKitTests/Open Groups/OpenGroupManagerSpec.swift
#	SessionMessagingKitTests/_TestUtilities/TestOnionRequestAPI.swift
#	SessionShareExtension/ShareNavController.swift
#	SessionSnodeKit/Jobs/GetSnodePoolJob.swift
#	SessionUtilitiesKit/Configuration.swift
#	SessionUtilitiesKit/Database/Utilities/Database+Utilities.swift
#	SessionUtilitiesKit/JobRunner/JobRunner.swift
#	SignalUtilitiesKit/Meta/SignalUtilitiesKit.h
#	SignalUtilitiesKit/Utilities/SSKAsserts.h
2023-07-18 10:02:51 +10:00
Morgan Pretty 38420997b0 Fixed a couple of bugs and made some performance tweaks
Cleaned up some duplicate poller logic (avoid going back to the main queue)
Updated the code to remove a profile image if a user sends a message which doesn't have a profile image (ie. they've explicitly removed it)
Fixed an issue where some more logic could incorrectly run in the DBWrite queue
Fixed a bug where the OpenGroupPoller could stop polling when getting an error
Fixed a bug where messages which had the same timestamp wouldn't get correctly marked as read when scrolling under the right circumstances
2023-07-10 17:56:58 +10:00
Morgan Pretty a7761697a9 More performance logging and some minor optimisations
Added code to throttle the 'markAsRead' logic when scrolling to 100ms
Added a launch counter so we will always get error logs on launch if the database is suspended
Updated the logging to also indicate whether it's the DBWrite queue
Moved a couple of Atomic mutations outside of DB write closure on the off chance they are blocking
2023-07-06 15:39:41 +10:00
Morgan Pretty 5e2e103ee1 Resolved the remaining known internal testing issues
Removed the 'immediatelyOnMain' extensions as they would break in some cases (eg. upstream errors with multiple 'receive(on:)' calls) resulting in logic running on unexpected threads
Updated the ReplaySubject to add subscribers in an Atomic just to be safe
Updated the code to remove the invalid open group when the user receives an error after joining
Fixed a bug with editing closed group members
Fixed broken unit tests
2023-06-29 12:11:54 +10:00
Morgan Pretty 53a5db0ea5 Fixed a number of issues found during internal testing
Added copy for an unrecoverable startup case
Added some additional logs to better debug ValueObservation query errors
Increased the pageSize to 20 on iPad devices (to prevent it immediately loading a second page)
Cleaned up a bunch of threading logic (try to avoid overriding subscribe/receive threads specified at subscription)
Consolidated the 'sendMessage' and 'sendAttachments' functions
Updated the various frameworks to use 'DAWRF with DSYM' to allow for better debugging during debug mode (at the cost of a longer build time)
Updated the logic to optimistically insert messages when sending to avoid any database write delays
Updated the logic to avoid sending notifications for messages which are already marked as read by the config
Fixed an issue where multiple paths could incorrectly get built at the same time in some cases
Fixed an issue where other job queues could be started before the blockingQueue finishes
Fixed a potential bug with the snode version comparison (was just a string comparison which would fail when getting to double-digit values)
Fixed a bug where you couldn't remove the last reaction on a message
Fixed the broken media message zoom animations
Fixed a bug where the last message read in a conversation wouldn't be correctly detected as already read
Fixed a bug where the QuoteView had no line limits (resulting in the '@You' mention background highlight being incorrectly positioned in the quote preview)
Fixed a bug where a large number of configSyncJobs could be scheduled (only one would run at a time but this could result in performance impacts)
2023-06-23 17:54:29 +10:00
Morgan Pretty 3b772b7f90 [WIP] started adding logic to ignore messages invalidated by config
Created a ConfigMessageReceiveJob just to clean up the logs a bit
Updated the poller to make any MessageReceive jobs dependant on any ConfigMessageReceive jobs which are created
Updated legacy groups to delete the group content when you are removed
Fixed an issue where the JobRunner wouldn't stop pending jobs while clearing data
Fixed another issue with the profile view in the message cell
2023-05-30 17:45:46 +10:00
Morgan Pretty 7ee84fe0d3 Cleaned up a bunch of code, added pinned and hidden handling
Added in logic to handle the 'hidden' state
Replaced the 'Group Created' message with an empty state
Cleaned up a bunch of boilerplate code
2023-03-02 17:52:37 +11:00
Morgan Pretty ff36b3eeab Resolved a number of TODOs and a couple of crashes
Updated the logic to always fetch all config messages (and not de-dupe them)
Updated the onboarding to fallback to fetch the legacy config if the there is no updated user config
Added sorting to config message processing to reduce the turn-around time for certain edge-cases to resolve
Removed some redundant code
Fixed a database re-entrancy crash
Fixed a crash when merging config messages
Fixed an issue which could occur by registering too many 'afterTransaction' closures
2023-02-21 17:10:02 +11:00
Morgan Pretty f30b383bb8 Updated to the latest version of libSession-util
Updated the SharedConfigMessage type to have a TTL of 30 days
Updated the SnodeAPI to have a 'poll' method to be more consistent with the OpenGroupAPI (it also does multiple things now so is cleaner)
Added logic to limit the number of config messages to be retrieved per poll
Added the 'ValidatableResponse' protocol to standardise SnodeAPI response validation
Added the libSession version to the logs
Fixed an issue where the user profile pic wouldn't get synced correctly due to memory going out of scope
Fixed some threading issues
Refactored the thread variants to follow the updated terminology (will think about refactoring other code areas later)
Cleaned up the Combine error handling
Started fixing broken unit tests
2023-02-20 12:56:48 +11:00
Morgan Pretty 8f3dcbc6be Sorted out a bunch of the config syncing logic
Updated the onboarding to attempt to retrieve the current user profile config and skip display name collection if it already exists
Updated the logic to get the snode pool and build paths immediately on launch even if the user hasn't been created yet (faster onboarding)
Removed the iOS-specific concurrent dual snode '/store' behaviour
Cleaned up the profile updating logic
Fixed an issue where the pollers could end up deadlocking the main thread if too many tried to start concurrently
2022-12-16 16:51:08 +11:00
Morgan Pretty f1e9412c7a Added in missing code changes unrelated to closed groups rebuild 2022-12-08 14:21:38 +11:00
Morgan Pretty c9fdee9f24 Work on the PromiseKit refactor
# Conflicts:
#	Session.xcodeproj/project.pbxproj
#	Session/Conversations/ConversationVC+Interaction.swift
#	Session/Home/Message Requests/MessageRequestsViewModel.swift
#	Session/Notifications/AppNotifications.swift
#	Session/Notifications/PushRegistrationManager.swift
#	Session/Notifications/SyncPushTokensJob.swift
#	Session/Notifications/UserNotificationsAdaptee.swift
#	Session/Settings/BlockedContactsViewModel.swift
#	Session/Settings/NukeDataModal.swift
#	Session/Settings/SettingsViewModel.swift
#	Session/Utilities/BackgroundPoller.swift
#	SessionMessagingKit/Database/Models/ClosedGroup.swift
#	SessionMessagingKit/File Server/FileServerAPI.swift
#	SessionMessagingKit/Open Groups/OpenGroupAPI.swift
#	SessionMessagingKit/Sending & Receiving/Message Handling/MessageReceiver+ClosedGroups.swift
#	SessionMessagingKit/Sending & Receiving/Message Handling/MessageReceiver+UnsendRequests.swift
#	SessionMessagingKit/Sending & Receiving/Message Handling/MessageSender+ClosedGroups.swift
#	SessionMessagingKit/Sending & Receiving/MessageSender+Convenience.swift
#	SessionMessagingKit/Sending & Receiving/MessageSender.swift
#	SessionMessagingKit/Sending & Receiving/Notifications/PushNotificationAPI.swift
#	SessionMessagingKit/Sending & Receiving/Pollers/ClosedGroupPoller.swift
#	SessionMessagingKit/Sending & Receiving/Pollers/CurrentUserPoller.swift
#	SessionMessagingKit/Sending & Receiving/Pollers/Poller.swift
#	SessionMessagingKit/Utilities/ProfileManager.swift
#	SessionSnodeKit/Networking/SnodeAPI.swift
#	SessionSnodeKit/OnionRequestAPI.swift
#	SessionUtilitiesKit/Networking/HTTP.swift
2022-12-07 15:50:58 +11:00
Morgan Pretty 1345e89809 Further config util logic
Removed the usage of the OWSAES256Key (using CryptoKit and raw data instead)
Removed the pre-compiled headers to speed up builds with minor changes (explicit imports instead)

# Conflicts:
#	Session.xcodeproj/project.pbxproj
#	SessionMessagingKit/Database/Models/ClosedGroup.swift
#	SessionMessagingKit/Protos/Generated/SNProto.swift
#	SessionMessagingKit/Protos/Generated/SessionProtos.pb.swift
#	SessionMessagingKit/Protos/SessionProtos.proto
#	SessionMessagingKit/Sending & Receiving/MessageSender.swift
#	SessionMessagingKit/Sending & Receiving/Pollers/CurrentUserPoller.swift
#	SessionMessagingKit/Utilities/ProfileManager.swift
#	SessionSnodeKit/Models/DeleteAllMessagesRequest.swift
#	SessionSnodeKit/Models/GetMessagesRequest.swift
#	SessionSnodeKit/Models/SendMessageRequest.swift
#	SessionSnodeKit/Types/SnodeAPINamespace.swift
2022-12-07 15:06:15 +11:00
Morgan Pretty d8fd3b35b4 Theming tweaks and bug fixes
Made a tweak to prevent some odd looking keyboard transitions when going to conversation settings
Updated the PagedDatabaseObserver to not call 'onChangeUnsorted' on the main thread (now we can generate the changeset on the background thread so there is less main thread work)
Fixed an issue where the most recently received message from the swarm could be removed from the swarm yet the app would still poll for it, resulting in the swarm always returning the oldest possible messages until the user sends a new one-to-one message
Fixed an issue where the initial scroll offset could be incorrect due to certain message types
Fixed an issue where the title view inside a conversation could jump when pushing to the conversation settings screen
Refactored a couple of ObjC functions to Swift as they were crashing (due to memory allocation?) hopefully this will fix it
Tweaked some DispatchQueue priorities to ensure PagedDatabaseObserver loading is prioritised
Updated buttons to use a standard convention for highlighted states
Updated the new conversation button to follow the new highlighted state convention
2022-10-14 17:09:38 +11:00
Morgan Pretty fc860e3957 Added fixes and defensive coding for a number of crashes
Fixed a crash which could occur due to multithreaded dependency access
Fixed a crash which could occur if the 'keyWindow' wasn't set on the LandingVC in time since we were force-unwrapping
Fixed a crash which could occur due to multithreaded swarm cache access
Fixed the broken unit tests
2022-08-30 09:45:40 +10:00
Morgan Pretty 7097853d58 A few bugs fixes and some optimisations
Fixed a bug where notifications could incorrectly appear for messages in the current thread
Fixed a bug where swiping left/right on images in the MediaDetailViewController could load images from other threads
Fixed a bug where the unread count wouldn't appear correctly when opening a conversation
Fixed a bug where the unread count on the conversation cell could get truncated
Fixed a bug where notifications weren't working correctly when the app is in the foreground
Fixed a bug where we weren't clearing the 'received X new messages' count when in the foreground
Fixed a bug where outgoing messages could get marked as read in a very specific case
Updated the "group notification" logic to only apply to Open Groups (and always doing it rather than just in the background)
Added a placeholder person icon when you have a closed group with a single member
Added a couple of indexes to improve the HomeVC database query performance (reduce launch time by ~15% in some cases)
Added a background task to give message sending the chance to complete when sending the app to the background
Removed an unneeded query from the HomeViewModel init (reduce launch time by ~10% in some cases)
Optimised one of the queries used to load the data needed for the conversation screen
2022-08-19 16:58:47 +10:00
Morgan Pretty 1224e539ea Reduced unneeded DB write operations and fixed a few minor UI bugs
Updated the database to better support the application getting suspended (0xdead10cc crash)
Updated the SOGS message handling to delete messages based on a new 'deleted' flag instead of 'data' being null
Updated the code to prevent the typing indicator from needing a DB write block as frequently
Updated the code to stop any pending jobs when entering the background (in an attempt to prevent the database suspension from causing issues)
Removed the duplicate 'Capabilities.Capability' type (updated 'Capability.Variant' to work in the same way)
Fixed a bug where a number of icons (inc. the "download document" icon) were the wrong colour in dark mode
Fixed a bug where the '@You' highlight could incorrectly have it's width reduced in some cases (had protection to prevent it being larger than the line, but that is a valid case)
Fixed a bug where the JobRunner was starting the background (which could lead to trying to access the database once it had been suspended)
Updated to the latest version of GRDB
Added some logic to the BackgroundPoller process to try and stop processing if the timeout is triggered (will catch some cases but others will end up logging a bunch of "Database is suspended" errors)
Added in some protection to prevent future deferral loops in the JobRunner
2022-08-05 17:10:01 +10:00
Morgan Pretty ecbded3819 Cleaned up the poller logic a bit 2022-08-04 18:09:03 +10:00
Morgan Pretty 8cf2a57fcc Renamed GRDBStorage to Storage (no use having the prefix anymore since the old DB is deprecated) 2022-07-01 13:08:45 +10:00
Morgan Pretty eeccfb47d5 Fixed all of the build errors from merge, migrated Call logic, started idBlinding migration and bug fixes
Fixed some broken file paths
Fixed a couple of bugs with closed groups
Fixed a few migration issues
Fixed a bug with the ProfilePictureView in open groups (was including the open parenthesis in the initials)
Migrated the Id Blinding changes to work with GRDB
Migrated the call logic to work with GRDB
Updated the code to work the with hard fork changes
2022-06-09 18:37:44 +10:00
Morgan Pretty 4dced25e85 Removed an initial request occurring when pushing the conversation screen (could hold up main thread)
Removed the old OWSPreferences file (everything left over seems to be unused)
2022-06-03 16:44:29 +10:00
Morgan Pretty af073657a2 Cleaned up received message handling and a few bugs with duplicate message handling
Updated the YDB to GRDB migrations to include some progress when importing swarms & interactions (ie. the slow parts we can't properly show progress for)
Changed the MessageReceiveJob into a MessageHandlingJob (when receiving a message we now parse and store everything immediately to avoid a number of weird edge-cases)
Fixed a bug where the Poller would drop a Snode when returning from the background because it's last request would generally time out
Fixed a few bugs with invalid attachments
Added the ability to retry downloading a failed attachment
Added back the search results limit
2022-06-03 15:47:16 +10:00
Morgan Pretty 59696f7d2c Removed all the old Storage classes and YapDB extensions
Removed the AppUpdateNag code (unused)
Removed the Mantle dependency
2022-06-01 17:41:20 +10:00
Morgan Pretty 8ff542405c Finished of the Conversation screen and JobQueue concurrency
Updated the migrations to indicate progress (Potential to base progress for the "processing" sections on the file size of the legacy database)
Updated the JobRunner to properly support concurrent queues for sending/receiving (other queues are still serial)
Added the typing indicator logic into the ConversationVC
Put code into SUKLegacy for connecting to the YDB database
Fixed a couple of minor UI bugs with the GalleryRailView
Updated the media gallery selection screen to use the appropriate system theme colouring (was painful to randomly swap from dark mode to like for one screen...)
Added an alert for when the database migration fails
Deleted the legacy migrations (manually applying any unapplied changes as part of the YDB to GRDB migration process)
2022-05-31 17:41:02 +10:00
Morgan Pretty 3514ed4f50 Updated the JobRunner to have multiple job queues (needs more testing)
Added a backoff to the Poller retry
Updated the "blocking" behaviour of the JobRunner
Tweaked the Job dependency handling to better handle orphaned dependencies
Fixed an issue where the Conversation screen wasn't observing database changes
2022-05-28 17:25:38 +10:00
Morgan Pretty aabf656d89 Finished off the MediaGallery logic
Updated the config message generation for GRDB
Migrated more preferences into GRDB
Added paging to the MediaTileViewController and sorted out the various animations/transitions
Fixed an issue where the 'recipientState' for the 'baseQuery' on the ConversationCell.ViewModel wasn't grouping correctly
Fixed an issue where the MediaZoomAnimationController could fail if the contextual info wasn't available
Fixed an issue where the MediaZoomAnimationController bounce looked odd when returning to the detail screen from the tile screen
Fixed an issue where the MediaZoomAnimationController didn't work for videos
Fixed a bug where the YDB to GRDB migration wasn't properly handling video files
Fixed a number of minor UI bugs with the GalleryRailView
Deleted a bunch of legacy code
2022-05-20 17:58:39 +10:00
Morgan Pretty 3f062c044c Added back the majority of the ConversationVC interactions
Removed some more legacy code
Added back logic similar to the pre-processing de-duping logic (was resulting in "unsent" messages reappearing)
Added a number of updated view files
2022-05-12 17:28:27 +10:00
Morgan Pretty 0db74ce1e3 Working on the MediaGallery and ClosedGroup handling
Fixed a couple of issues around the duplicate messages handling
Fixed a few issues with ClosedGroup polling and ClosedGroup control message handling
Started working through updating the MediaGallery
2022-05-08 22:01:39 +10:00
Morgan Pretty b541666ef0 Got the ability to send message working again and other tweaks
Tested and fixed a couple of issues with the disappearingMessages job
Added a simple dependency system for jobs (primarily for the AttachmentUploadJob, but will likely be others later)
Setup the AttachmentUploadJob again (looks like there are cases which use it)
Prevented a possible infinite job deferral loop from causing the app to crash (the loop is still technically possible but the app will continue to run now)
Updated the interactions unique constraints based on testing and discussions around how the serverHash works
Deleted the legacy ReadReceipt handling (now managed via the 'interaction.wasRead' flag and 'SendReadReceiptsJob')
Deleted the unused SSKIncrementingIdFinder
2022-05-06 12:44:26 +10:00
Morgan Pretty 3baeb981d9 Further work on the JobRunner
Moved the JobRunner into SessionUtilitiesKit so it can be used by SessionSnodeKit
Exposed a 'sharedLokiProject' value on UserDefaults to remove the hard-coded group name used everywhere
Added "blocking" job support for 'OnLaunch' and 'OnActive' jobs to the JobRunner (will retry until it succeeds)
Added the UpdateProfilePicture and RetrieveDefaultOpenGroupRooms jobs
2022-04-27 10:48:54 +10:00
Morgan Pretty ed9f4ea6c6 Fixed a few closed group and job issues
Fixed a few job migration issues
Fixed an issue with the closed group key pair management (wasn't storing keys correctly)
Refactored the OWSSound (now Preferences.Sound)
Added the logic for the AttachmentDownloadJob and enabled jobs to be cascade deleted via interactions
Optimised the HomeViewModel database observation query (fetch specific columns so changes outside those don't trigger updates)
Updated to the latest GRDB (ran into a deadlock which should be fixed in a newer version)
2022-04-22 18:47:11 +10:00
Morgan Pretty 11231599db Further work on migrations and message pipeline refactoring
Refactored the AppDelegate from Objective C to Swift
Updated the HomeVC to use GRDB
Refactored a number of the Job types to be driven via GRDB and the new JobRunner
Fixed a bug where the LinkPreviewView wouldn't render correctly in dark mode
2022-04-21 16:42:35 +10:00
Morgan Pretty a1b4554cdb Migrated the SessionSnodeKit from YapDatabase to GRDB
Changed the min OS version to iOS 13.0 (support for 'Identifiable')
Removed the alternate approaches to fetching the userKeyPair and userPublicKeyHexString (no consistently routed through the caching method)
Migrated the 'OWSIdentityManager' logic to use the new 'Identity' type
Added the 'Setting' table and got the pattern working fairly nicely (unfortunately there isn't a good way to avoid key collision without proper enums)
Updated the SessionSnodeKit to migration it's data from YDB to GRDB
Updated the SessionSnodeKit to use GRDB throughout it's logic
2022-04-01 17:22:45 +11:00
Morgan Pretty e65682ae9b Merge branch 'dev' into feature/database-refactor
# Conflicts:
#	Session.xcodeproj/project.pbxproj
#	Session/Backups/OWSBackup.m
#	Session/Backups/OWSBackupImportJob.m
#	SessionSnodeKit/SnodeAPI.swift
2022-03-31 12:09:04 +11:00
Morgan Pretty 529e416dd1 Started work on GRDB logic and migrations
Setup a migration pattern
Setup the database configuration and security
Started defining the database schema
Started working on the migrations for SessionSnodeKit
2022-03-31 11:47:09 +11:00
Morgan Pretty b90904ebbd Updated the code to only update the last message hash once the MessageReceiveJobs have been created 2022-03-22 14:09:47 +11:00
ryanzhao cd97b9c3e8 minor fix on poller threading 2022-02-23 12:47:44 +11:00
ryanzhao b2ab984586 refactor on timer and polling threading 2022-02-22 14:01:48 +11:00
Ryan Zhao d21c5329e3 further improvements to move poller into background threads 2022-02-21 17:01:33 +11:00
Ryan Zhao 16c9b7793a move pollers to background threads 2022-02-21 13:27:44 +11:00
Ryan Zhao e8e120666b move message poller to back ground thread 2022-02-17 16:38:52 +11:00
Ryan Zhao 08360273e6 save server hash for message receive jobs 2021-08-02 15:39:10 +10:00
Niels Andriesse bd04775cbf Implement open group invitations UI 2021-05-07 13:10:21 +10:00
Niels Andriesse 40d199a90b Clean up SnodeAPI 2020-12-18 15:17:23 +11:00
Niels Andriesse efe8f1c8bb Clean 2020-12-07 16:00:21 +11:00
Niels Andriesse 438bbccdfa Fix background polling 2020-12-07 11:21:24 +11:00
nielsandriesse 5dfd4b1965 Restructure 2020-11-25 16:15:16 +11:00
Renamed from SignalUtilitiesKit/Messaging/Sending & Receiving/Poller.swift (Browse further)