Allison Lortie
2016-12-23 15:32:55 UTC
hi,
I apologise in advance for the novel below.
Take a look in your ~/.local/share/. It's probably a mess.
We're basically using this directory for two things right now, and I
think that only one of them is appropriate.
1) We are using it as the "user's layer" for XDG_DATA_DIRS
2) We are using it as a dumping ground for application state
It's worth mentioning that usage #2 is not specifically endorsed by the
specification itself. It says that "XDG_DATA_HOME defines the base
directory relative to which user specific data files should be stored"
and goes to quite some lengths to talk about how it is basically a
higher-priority "user-specific" version of XDG_DATA_DIRS. It never
talks about it as being an appropriate place to dump writable
application state data.
But here we are. Everyone does this, because they think that ~/.config/
is not an appropriate place to be putting stuff that's not really
configuration (and I agree).
Further: due to the dual usage of XDG_DATA_HOME for XDG_DATA_DIRS type
stuff (applications/, icons/, fonts/, etc.) and random application
state, some people (myself included) hesitate to fully embrace it as
"the top XDG_DATA_DIR". GLib, for example, still has two separate APIs
for this variable, and never treats XDG_DATA_HOME as the top layer of
XDG_DATA_DIRS.
For some time (years) I've wanted to clean up this mess and introduce a
proper, documented, place for application state information. In fact,
I've received many pleas for this from several ISVs since some years
ago, who port an app from another platform, and just want one directory
that they can use for everything. Since we've never documented what
this place is, some are (rightly) apprehensive about just using
~/.local/share/ and contributing to the problem.
So let's try this:
I'd like to introduce the XDG_STATE_HOME variable. It would default to
~/.var/. Just as ~/.local/share is the user's version of
/usr/local/share, ~/.var/ is the user's version of /var. Applications
that wish to store unstructured state data (bookmarks, cookies, game
save files, internal databases, etc.) should create a subdirectory under
this directory thus:
XDG_STATE_HOME + "/app/" + app_id
and store their state there.
For example, you might see ~/.var/app/org.gnome.gedit/.
We could also imagine ~/.var/lib/ subdirectories for use by system
components (read: not apps) or "well known" files, such as things like
recently-used.xbel.
Meanwhile, I think we should strengthen the language in the spec about
how XDG_DATA_HOME should absolutely be considered the top layer of
XDG_DATA_DIRS, and that compliant applications and libraries should
always allow the user's verion of a file found here to take precedence
over those found at the system level. The expected use for this
directory would then be *exclusively* for things like icons, desktop
files, gsettings schemas, fonts, etc.
One day we might also dream about moving the hilariously inappropriate
~/.local/share/Trash/ to ~/.var/trash/ or even ~/.cache/ to
~/.var/cache/. See [1] for information on back-compat.
As for migration of application data: it's my opinion that we could be
somewhat neutral on this point. We certainly will not automatically
move any application data by a specification change, or even a (eg.
GLib) library change. Applications would have the choice to follow the
new convention or not, but if they did, we should maybe give some
pointers on how to handle the migration. Again, see [1].
Next on the wish list, I guess, would be introduction of language into
the spec to suggest that apps also only create properly named
(org.gnome.gedit style) directories under ~/.config/ as well, again
using [1]. This is a separate topic, though.
I welcome everyone's thoughts on this topic. I expect that we will be
discussing this well into the new year. :)
Allison
[1] Backwards compatibility is important for two reasons:
1. applications which want to keep their data from older versions
2. "common" directories which will be accessed by apps and libraries
under both the new and the old conventions
I propose the following trick to handle both cases. Normally, an app
computes the effective 'dirname' and then does something like:
dirname = (compute);
if not exists dirname:
mkdir_with_parents(dirname)
...use dirname...
Here is a nice trick for "moving" to a new place:
new_dirname = (compute); // the new location
if not exists new_dirname:
// make the parents, but not the dir itself
mkdir_with_parents(parent(new_dirname))
// check if the old directory exists:
old_dirname = (compute); // the former location
if exists old_dirname:
make_relative_symlink(old_dirname, new_dirname);
else:
mkdir(new_dirname)
...use new_dirname...
The effect is that "new" users in a clean home directory (or those who
manually take steps to "clean house") would start using only the new
paths, while existing users with data from before the transition would
have their files left in place, and the new version of the app would
follow a symlink. This has a lot of nice properties:
- we don't annoy the user by moving data around
- backup exclude lists continue to work
- the described algorithm is race-free
- the 'old' apps don't see any symlink, so they won't break because of
it
- after the 1st run (ie: dir exists) it's just as efficient as before
There is a drawback: one claim of this system is that it allows 'new'
and 'old' app versions to continue to share the directory. If the 'new'
version is the first to start, however, the new directory will be
created, and the old directory will be completely left alone. When the
old version starts, it will create the old version, as a completely
separate directory. I consider this case to be rare for application
data, since downgrades and multiple versions are uncommon.
This problem might be an issue if we decide to move directories like
cache or trash, since it will be quite common to have a mix of
applications following 'new' and 'old' conventions installed. If the
'new' apps run first, we end up with two separate directories. For
cache, I guess this is no big problem (since sharing cached data is
rare, and even in the case of things like thumbnails, failing to share
is not a huge problem). For trash, we might have to be more careful,
however, and it may be worth introducing specific language to the spec.
I apologise in advance for the novel below.
Take a look in your ~/.local/share/. It's probably a mess.
We're basically using this directory for two things right now, and I
think that only one of them is appropriate.
1) We are using it as the "user's layer" for XDG_DATA_DIRS
2) We are using it as a dumping ground for application state
It's worth mentioning that usage #2 is not specifically endorsed by the
specification itself. It says that "XDG_DATA_HOME defines the base
directory relative to which user specific data files should be stored"
and goes to quite some lengths to talk about how it is basically a
higher-priority "user-specific" version of XDG_DATA_DIRS. It never
talks about it as being an appropriate place to dump writable
application state data.
But here we are. Everyone does this, because they think that ~/.config/
is not an appropriate place to be putting stuff that's not really
configuration (and I agree).
Further: due to the dual usage of XDG_DATA_HOME for XDG_DATA_DIRS type
stuff (applications/, icons/, fonts/, etc.) and random application
state, some people (myself included) hesitate to fully embrace it as
"the top XDG_DATA_DIR". GLib, for example, still has two separate APIs
for this variable, and never treats XDG_DATA_HOME as the top layer of
XDG_DATA_DIRS.
For some time (years) I've wanted to clean up this mess and introduce a
proper, documented, place for application state information. In fact,
I've received many pleas for this from several ISVs since some years
ago, who port an app from another platform, and just want one directory
that they can use for everything. Since we've never documented what
this place is, some are (rightly) apprehensive about just using
~/.local/share/ and contributing to the problem.
So let's try this:
I'd like to introduce the XDG_STATE_HOME variable. It would default to
~/.var/. Just as ~/.local/share is the user's version of
/usr/local/share, ~/.var/ is the user's version of /var. Applications
that wish to store unstructured state data (bookmarks, cookies, game
save files, internal databases, etc.) should create a subdirectory under
this directory thus:
XDG_STATE_HOME + "/app/" + app_id
and store their state there.
For example, you might see ~/.var/app/org.gnome.gedit/.
We could also imagine ~/.var/lib/ subdirectories for use by system
components (read: not apps) or "well known" files, such as things like
recently-used.xbel.
Meanwhile, I think we should strengthen the language in the spec about
how XDG_DATA_HOME should absolutely be considered the top layer of
XDG_DATA_DIRS, and that compliant applications and libraries should
always allow the user's verion of a file found here to take precedence
over those found at the system level. The expected use for this
directory would then be *exclusively* for things like icons, desktop
files, gsettings schemas, fonts, etc.
One day we might also dream about moving the hilariously inappropriate
~/.local/share/Trash/ to ~/.var/trash/ or even ~/.cache/ to
~/.var/cache/. See [1] for information on back-compat.
As for migration of application data: it's my opinion that we could be
somewhat neutral on this point. We certainly will not automatically
move any application data by a specification change, or even a (eg.
GLib) library change. Applications would have the choice to follow the
new convention or not, but if they did, we should maybe give some
pointers on how to handle the migration. Again, see [1].
Next on the wish list, I guess, would be introduction of language into
the spec to suggest that apps also only create properly named
(org.gnome.gedit style) directories under ~/.config/ as well, again
using [1]. This is a separate topic, though.
I welcome everyone's thoughts on this topic. I expect that we will be
discussing this well into the new year. :)
Allison
[1] Backwards compatibility is important for two reasons:
1. applications which want to keep their data from older versions
2. "common" directories which will be accessed by apps and libraries
under both the new and the old conventions
I propose the following trick to handle both cases. Normally, an app
computes the effective 'dirname' and then does something like:
dirname = (compute);
if not exists dirname:
mkdir_with_parents(dirname)
...use dirname...
Here is a nice trick for "moving" to a new place:
new_dirname = (compute); // the new location
if not exists new_dirname:
// make the parents, but not the dir itself
mkdir_with_parents(parent(new_dirname))
// check if the old directory exists:
old_dirname = (compute); // the former location
if exists old_dirname:
make_relative_symlink(old_dirname, new_dirname);
else:
mkdir(new_dirname)
...use new_dirname...
The effect is that "new" users in a clean home directory (or those who
manually take steps to "clean house") would start using only the new
paths, while existing users with data from before the transition would
have their files left in place, and the new version of the app would
follow a symlink. This has a lot of nice properties:
- we don't annoy the user by moving data around
- backup exclude lists continue to work
- the described algorithm is race-free
- the 'old' apps don't see any symlink, so they won't break because of
it
- after the 1st run (ie: dir exists) it's just as efficient as before
There is a drawback: one claim of this system is that it allows 'new'
and 'old' app versions to continue to share the directory. If the 'new'
version is the first to start, however, the new directory will be
created, and the old directory will be completely left alone. When the
old version starts, it will create the old version, as a completely
separate directory. I consider this case to be rare for application
data, since downgrades and multiple versions are uncommon.
This problem might be an issue if we decide to move directories like
cache or trash, since it will be quite common to have a mix of
applications following 'new' and 'old' conventions installed. If the
'new' apps run first, we end up with two separate directories. For
cache, I guess this is no big problem (since sharing cached data is
rare, and even in the case of things like thumbnails, failing to share
is not a huge problem). For trash, we might have to be more careful,
however, and it may be worth introducing specific language to the spec.