Recent comments posted to this site:
@arseny-n the misctemp directory does not normally contain anything, or only temp files in use by the currently running git-annex process for a short amount of time. The only way I know of that it can get files piled up in it is when you kill the git-annex process while it's using such a file.
It's always safe to delete the files in misctemp as long as git-annex is not running. Also, the names of the files should give a pretty good clue about what git-annex was using the file for. For example "jlog" files are used for staging the journal.
What if the remote wants to use some feature like NOTE, but can still manage to work when an old git-annex does not support it? Hard bumping the VERSION cannot support that. If the remote requires to be able to use NOTE and sees it cannot, it can still throw an error.
There are a bunch of requests in the protocol that are optional for the remote to support; git-annex deals with remotes that don't support them in better ways than throwing up its hands because the special remote is too old. It's very good that the protocol allowed adding those extensions without bumping a version. The protocol is less extensible when it comes replies and other messages sent by the special remote, and I want to get the same extensibility for those.
Changing VERSION would prevent any older versions of git-annex from working with that external special remote, since they would reject the unknown version. (The current parsing of VERSION also happens to preclude adding some fields after the number.)
I still do not get it, sorry -- If there is an older git-annex, and a special remote requests some higher VERSION (thus stating that it needs some features older git-annex does not support), IMHO it would be perfectly fine to fail to use that remote since it wouldn't be usable anyways with that older git-annex (i.e. require some special features it does not provide). If special remote does not need any feature not present in version 1
, it (like all of them ATM) could still keep requesting VERSION 1
thus staying compatible with whatever old git-annex is out there.
Changing VERSION would prevent any older versions of git-annex from working with that external special remote, since they would reject the unknown version. (The current parsing of VERSION also happens to preclude adding some fields after the number.)
Since it seems completely possible to make the protocol be changed in a way that is backwards compatible both ways, while still letting new features to be used, I'd rather reserve changing VERSION for whatever future thing needs a full breaking bump.
joey wrote:
The protocol VERSION is picked by the special remote, it's not negotiated.
VERSION
is provided to git-annex by the special remote to git-annex process. There is no need to 'negotiate' anything - you could make git-annex understand either of:
higher
VERSION
, e.g.VERSION 2
which would support some new features which that special remote would need. If parent git-annex is old/doesn't support that version - would fail and demand git annex upgradeVERSION 6.20171124
(where6.20171124
is an example of git-annex version) so if git-annex parent process is older than that it could provide a meaningful message thatgit annex >= 6.20171124
is needed
VERSION 1 feature1 feature2 ...
where those features could be the ones needed (e.g.INFO_MSG
for recent addition). And if parent git-annex doesn't know/support any particular feature, it could fail and inform user that a new annex with that feature support is needed.
In either of those cases the custom special remotes page could outline added features/versions of git-annex supporting them, so may be even those above error messages could point to it.
Overall, it is just a minor change to be done on git-annex side while allowing for clear(er) specification, and I do not see any need for actual "negotiation" -- features are either supported or not by the parent process.
I see that you manually add the first repo as a remote of the external drive, and the external drive as a remote of the original.
What about when you add a second external drive? Should you then be adding the first two repos as remotes of the new one, and then go to the first two ones and add the new drive as a remote of them too? Doesn't this permutation scale out of control when you add a new drive to, say, a git annex made of 10 disks, 20 disks, etc?
Why is .git/annex/misctmp so large ? Currently I use git annex to manage pytorch models, basically I have a large amount (1500 folder) of 4 Kilobytes files, some files are are bigger, misctmp occupies 6.2 GB, is it ok ?
PS. Sorry, if I write this here but failed to post to the forum.