@powersync/attachments deletes synced items from storage when greater than cacheLimit

[riisi]

My expectation of the way cache should work (it's a little unclear from the docs) is that the cache is a local cache, and @powersync/attachments should manage the size. The full set of current (not-orphaned/archived) data should remain on remote.

In testing, I found that if cacheLimit is set to a low value, e.g., 2, then only 2 items are preserved in (remote) storage.
So the cacheLimit effectively applies to both local and remote files.

Inspecting the source seems to confirm this -

The delete method deletes from both local and remote:

powersync-js/packages/attachments/src/AbstractAttachmentQueue.ts

Lines 224 to 250 in ba72a58

	async delete(record: AttachmentRecord, tx?: Transaction): Promise<void> {
	const deleteRecord = async (tx: Transaction) => {
	await tx.execute(
	`DELETE
	FROM ${this.table}
	WHERE id = ?`,
	[record.id]
	);
	};
	if (tx) {
	await deleteRecord(tx);
	} else {
	await this.powersync.writeTransaction(deleteRecord);
	}
	const localFilePathUri = this.getLocalUri(record.local_uri || this.getLocalFilePathSuffix(record.filename));
	try {
	// Delete file on storage
	await this.storage.deleteFile(localFilePathUri, {
	filename: record.filename
	});
	} catch (e) {
	this.logger.error(e);
	}
	}

delete() is called from expire() to delete the oldest synced or archived attachment:

powersync-js/packages/attachments/src/AbstractAttachmentQueue.ts

Lines 513 to 531 in ba72a58

	async expireCache() {
	const res = await this.powersync.getAll<AttachmentRecord>(`SELECT * FROM ${this.table}
	WHERE
	state = ${AttachmentState.SYNCED} OR state = ${AttachmentState.ARCHIVED}
	ORDER BY
	timestamp DESC
	LIMIT 100 OFFSET ${this.options.cacheLimit}`);
	if (res.length == 0) {
	return;
	}
	this.logger.debug(`Deleting ${res.length} attachments from cache...`);
	await this.powersync.writeTransaction(async (tx) => {
	for (const record of res) {
	await this.delete(record, tx);
	}
	});
	}

Am I missing something here?

[hunterbmiller]

I think that whether or not your the data is deleted remotely depends on how you implement your storage method, since powersync leaves it up to us to implement.

I've recently been implementing this myself, and have it such that the files aren't deleted remotely.

[stevensJourney]

To my knowledge, the JS attachment package does not support deleting attachments on the remote service. IIRC, it only supports uploads and downloads.

From looking at the code, the expireCache deletes the records and local files for synced and archived attachments. This specifically does not delete records for pending uploads - these should continue to be uploaded. While any synced records should be resynced after the delete.

FWIW the definition of the cache is a bit odd in this version. In other versions of the attachment helpers we define the cache as mentioned in your post - only archived data is treated as being present in the cache (we don't delete synced records).

[stevensJourney]

Update: this will be fixed in #735. Where the attachments helpers have been refactored to match our other SDK implementations.

The logic now will only consider archived attachments as "cached". Keeping archived attachments in a cache allows re-using the potentially downloaded asset if the attachment is restored / referenced again from the watched attachments query.

This now only limits the amount of archived attachments which are stored on the device at any time. This means any in-use attachments won't ever be automatically deleted.