volatile config data and XDG Base Directory spec

[Thomas Koch]

Hi,

I wanted to ask this since some months and the mplayer discussion reminded me:

Some applications store "volatile" config data or "convenience data" like:

- last window position
- recently opened file
- last time application was run
- ... you name it

Most annoyingly these applications create config files on first run even if I
won't ever run those apps again! So my home folder accumulates many worthless
config files.

There are many examples of this in the .kde folder.

I don't like to really call this data "configuration". It's not intentionally
set by the user. It has lot less value than "real hand crafted configuration"
(think .vimrc, .emacs, .zsh, .gnupg/gpg.conf, .ssh/config).

Do you have an idea for a name for such transient settings? Maybe "state"?

I'd really like to have an additional category besides CACHE, DATA, CONFIG for
this kind of settings. Having config and state mixed is mostly annoying if

- one keeps config in a VCS (git),
- an application isn't used anymore but the state data remains
- configuration is somehow managed centrally and the application doesn't read
configuration from /etc

Best regards,

Thomas Koch, http://www.koch.ro

[Thomas Kluyver]

Post by Thomas Koch
- last window position
- recently opened file
- last time application was run
- ... you name it
Most annoyingly these applications create config files on first run even if I
won't ever run those apps again! So my home folder accumulates many worthless
config files.

You could kind of call this data a cache - the user could easily move the
window to their preferred position, find a file they've recently used
again, and so on. The backup test - how much do you care if you lose it -
also suggests it could go under cache.

Thomas

[Thomas Koch]

Post by Thomas Koch
- last window position
- recently opened file
- last time application was run
- ... you name it
Most annoyingly these applications create config files on first run even if
I won't ever run those apps again! So my home folder accumulates many
worthless config files.
There are many examples of this in the .kde folder.
I don't like to really call this data "configuration". It's not
intentionally set by the user. It has lot less value than "real hand
crafted configuration" (think .vimrc, .emacs, .zsh, .gnupg/gpg.conf,
.ssh/config).
Do you have an idea for a name for such transient settings? Maybe "state"?
I'd really like to have an additional category besides CACHE, DATA, CONFIG
for this kind of settings. Having config and state mixed is mostly
annoying if
- one keeps config in a VCS (git),
- an application isn't used anymore but the state data remains
- configuration is somehow managed centrally and the application doesn't
read configuration from /etc

We've had this discussion again at Debconf[1] and discovered that several
people reported this issue on different channels. We agreed that it would make
sense to add a third category STATE which goes into a place of its own.

We don't think that STATE is the same as CACHE. The "backup test" is not a
good indicator. I could live with loosing the state data in an accident. But I
keep my .cache folder in a tmpfs so that it does not persist across reboots. I
would not like my STATE data to go away with every reboot.

[1] http://debconf13.debconf.org/

Regards, Thomas Koch

[Jerome Leclanche]

Post by Thomas Koch

Post by Thomas Koch
Some applications store "volatile" config data or "convenience data"
- last window position
- recently opened file
- last time application was run
- ... you name it
Most annoyingly these applications create config files on first run even

if

Post by Thomas Koch
I won't ever run those apps again! So my home folder accumulates many
worthless config files.
There are many examples of this in the .kde folder.
I don't like to really call this data "configuration". It's not
intentionally set by the user. It has lot less value than "real hand
crafted configuration" (think .vimrc, .emacs, .zsh, .gnupg/gpg.conf,
.ssh/config).
Do you have an idea for a name for such transient settings? Maybe

"state"?

Post by Thomas Koch
I'd really like to have an additional category besides CACHE, DATA,

CONFIG

Post by Thomas Koch
for this kind of settings. Having config and state mixed is mostly
annoying if
- one keeps config in a VCS (git),
- an application isn't used anymore but the state data remains
- configuration is somehow managed centrally and the application doesn't
read configuration from /etc

We've had this discussion again at Debconf[1] and discovered that several
people reported this issue on different channels. We agreed that it would make
sense to add a third category STATE which goes into a place of its own.
We don't think that STATE is the same as CACHE. The "backup test" is not a
good indicator. I could live with loosing the state data in an accident. But I
keep my .cache folder in a tmpfs so that it does not persist across reboots. I
would not like my STATE data to go away with every reboot.

Realistically, that's not feasible. The basedir spec is having a hard
enough time being adopted as it is. Say we introduce XDG_STATE_HOME:
- Plenty of apps still create config files on first run, because that's
just what they do.
- One more dotdir for $HOME, at very little benefit.
- More confusion for developers who want to use the basedir spec. "Does
this count as state?" is a question they now have to ask themselves for
every single setting.
- Most GUI apps use their toolkit's Settings class. State would have to
either be managed separately or be implemented within the toolkit and
specified declaratively somehow. Apart from window geometry, nearly none of
it would be handled automatically; for the rest it would just be more work.
-> At this point, you're better off saving window geometry state in your
WM config.
-> Falling short, there are ways to manage and clean ~/.config
programmatically; that's one of the big bonuses of having this spec.

I've been a big evangelist of the basedir spec and I'm more often than not
met with developers who (understandably) flat out refuse to split their
config file into a dir, or multiple dirs ("cache is not config", ...). If
on top of that you start telling them "But this type of config goes here!",
they'll laugh at you.

In short: -1

Post by Thomas Koch
[1] http://debconf13.debconf.org/
Regards, Thomas Koch
_______________________________________________
xdg mailing list
http://lists.freedesktop.org/mailman/listinfo/xdg

J. Leclanche

[David Faure]

Post by Jerome Leclanche

Post by Thomas Koch
Some applications store "volatile" config data or "convenience data"
- last window position
- recently opened file
- last time application was run
- ... you name it
Most annoyingly these applications create config files on first run even

if

Post by Thomas Koch
I won't ever run those apps again! So my home folder accumulates many
worthless config files.
There are many examples of this in the .kde folder.
I don't like to really call this data "configuration". It's not
intentionally set by the user. It has lot less value than "real hand
crafted configuration" (think .vimrc, .emacs, .zsh, .gnupg/gpg.conf,
.ssh/config).

That stuff isn't in XDG config anyway, so the argument doesn't really hold :-)
But yeah there's other stuff in there which is real config
(e.g. LDAP server for kaddressbook).

Post by Jerome Leclanche
[...]
Realistically, that's not feasible. The basedir spec is having a hard
- Plenty of apps still create config files on first run, because that's
just what they do.
- One more dotdir for $HOME, at very little benefit.
- More confusion for developers who want to use the basedir spec. "Does
this count as state?" is a question they now have to ask themselves for
every single setting.
- Most GUI apps use their toolkit's Settings class. State would have to
either be managed separately or be implemented within the toolkit and
specified declaratively somehow. Apart from window geometry, nearly none of
it would be handled automatically; for the rest it would just be more work.
-> At this point, you're better off saving window geometry state in your
WM config.
-> Falling short, there are ways to manage and clean ~/.config
programmatically; that's one of the big bonuses of having this spec.
I've been a big evangelist of the basedir spec and I'm more often than not
met with developers who (understandably) flat out refuse to split their
config file into a dir, or multiple dirs ("cache is not config", ...). If
on top of that you start telling them "But this type of config goes here!",
they'll laugh at you.
In short: -1

Right.

On the other hand, configuration frameworks could be a bit more tidy in the
way they store settings. I.e. without changing the spec, KDE apps (which were
the initial complaint in this thread) could store state in separate files in a
subdir of the config home.

The reason why "all KDE apps do this" is because it's done centrally
(KMainWindow), so this could easily be changed in one place, and it would
affect all KDE apps. Other frameworks might not do this (saving window state
into the config file), so they might not need this spec change, this is why
I'm suggesting to fix it in the implementation only.
Same for "recently opened files", KFileDialog does that, could be a separate
config file too.

This would fix "keeping the real config in git" (just exclude the subdir).

OTOH it would incur a small performance penalty, opening and parsing more
files than currently. But the files would be smaller, and e.g. one wouldn't
parse the recent files until opening the file dialog or the recent files
submenu.

Post by Jerome Leclanche
- an application isn't used anymore but the state data remains

Not much anyone can do about that, in any solution, but at least the real
config wouldn't be polluted.

Post by Jerome Leclanche
- configuration is somehow managed centrally and the application doesn't
read configuration from /etc

And I don't really understand what was meant here.

[François Revol]

Post by David Faure

Post by Thomas Koch
Some applications store "volatile" config data or "convenience data"
- last window position

In Haiku at least, this is the kind of thing that is stored as an xattr
on the relevant file/folder instead usually...
Of course, Linux still hasn't come to a point where xattr is considered
mandatory, shame on you :p

Post by David Faure

Post by Thomas Koch
- recently opened file
- last time application was run
- ... you name it
Most annoyingly these applications create config files on first run even if
I won't ever run those apps again! So my home folder accumulates many
worthless config files.
There are many examples of this in the .kde folder.
I don't like to really call this data "configuration". It's not
intentionally set by the user. It has lot less value than "real hand
crafted configuration" (think .vimrc, .emacs, .zsh, .gnupg/gpg.conf,
.ssh/config).

That stuff isn't in XDG config anyway, so the argument doesn't really hold :-)
But yeah there's other stuff in there which is real config
(e.g. LDAP server for kaddressbook).

I'd say it's rather session-specific indeed.

François.

[Dieter Plaetinck]

On Tue, 13 Aug 2013 14:05:08 +0100

Post by Jerome Leclanche

Post by Thomas Koch

Post by Thomas Koch
Some applications store "volatile" config data or "convenience data"
- last window position
- recently opened file
- last time application was run
- ... you name it
Most annoyingly these applications create config files on first run even

if

Post by Thomas Koch
I won't ever run those apps again! So my home folder accumulates many
worthless config files.
There are many examples of this in the .kde folder.
I don't like to really call this data "configuration". It's not
intentionally set by the user. It has lot less value than "real hand
crafted configuration" (think .vimrc, .emacs, .zsh, .gnupg/gpg.conf,
.ssh/config).
Do you have an idea for a name for such transient settings? Maybe

"state"?

Post by Thomas Koch
I'd really like to have an additional category besides CACHE, DATA,

CONFIG

Post by Thomas Koch
for this kind of settings. Having config and state mixed is mostly
annoying if
- one keeps config in a VCS (git),
- an application isn't used anymore but the state data remains
- configuration is somehow managed centrally and the application doesn't
read configuration from /etc

We've had this discussion again at Debconf[1] and discovered that several
people reported this issue on different channels. We agreed that it would make
sense to add a third category STATE which goes into a place of its own.
We don't think that STATE is the same as CACHE. The "backup test" is not a
good indicator. I could live with loosing the state data in an accident. But I
keep my .cache folder in a tmpfs so that it does not persist across reboots. I
would not like my STATE data to go away with every reboot.

Realistically, that's not feasible. The basedir spec is having a hard
- Plenty of apps still create config files on first run, because that's
just what they do.
- One more dotdir for $HOME, at very little benefit.
- More confusion for developers who want to use the basedir spec. "Does
this count as state?" is a question they now have to ask themselves for
every single setting.
- Most GUI apps use their toolkit's Settings class. State would have to
either be managed separately or be implemented within the toolkit and
specified declaratively somehow. Apart from window geometry, nearly none of
it would be handled automatically; for the rest it would just be more work.
-> At this point, you're better off saving window geometry state in your
WM config.
-> Falling short, there are ways to manage and clean ~/.config
programmatically; that's one of the big bonuses of having this spec.
I've been a big evangelist of the basedir spec and I'm more often than not
met with developers who (understandably) flat out refuse to split their
config file into a dir, or multiple dirs ("cache is not config", ...). If
on top of that you start telling them "But this type of config goes here!",
they'll laugh at you.

I think the xdg basedir spec has good ideals, and sure, it can take some work to adopt it properly. But this subject comes up often enough to say it should be dealt with properly.
Having a spec that takes care of config, data, cache but not state (at least not in a proper way) makes the spec incomplete and what's the point of that?
Keeping the spec 3/4-complete as a "favor" to developers who are short on time/interest seems lame; I would rather finish the spec and if devs don't feel like implementing it all the way, they should at least accept patches from people who do.

[Dieter Plaetinck]

On Thu, 22 Aug 2013 08:27:21 -0700

Post by Thomas Koch

Post by Dieter Plaetinck
On Tue, 13 Aug 2013 14:05:08 +0100

Post by Jerome Leclanche

Post by Thomas Koch

Post by Thomas Koch
Some applications store "volatile" config data or "convenience data"
- last window position
- recently opened file
- last time application was run
- ... you name it
Most annoyingly these applications create config files on first run

even

Post by Dieter Plaetinck

Post by Jerome Leclanche

Post by Thomas Koch
if

Post by Thomas Koch
I won't ever run those apps again! So my home folder accumulates

many

Post by Dieter Plaetinck

Post by Jerome Leclanche

Post by Thomas Koch

Post by Thomas Koch
worthless config files.
There are many examples of this in the .kde folder.
I don't like to really call this data "configuration". It's not
intentionally set by the user. It has lot less value than "real hand
crafted configuration" (think .vimrc, .emacs, .zsh, .gnupg/gpg.conf,
.ssh/config).
Do you have an idea for a name for such transient settings? Maybe

"state"?

Post by Thomas Koch
I'd really like to have an additional category besides CACHE, DATA,

CONFIG

Post by Thomas Koch
for this kind of settings. Having config and state mixed is mostly
annoying if
- one keeps config in a VCS (git),
- an application isn't used anymore but the state data remains
- configuration is somehow managed centrally and the application

doesn't

Post by Dieter Plaetinck

Post by Jerome Leclanche

Post by Thomas Koch

Post by Thomas Koch
read configuration from /etc

We've had this discussion again at Debconf[1] and discovered that

several

Post by Dieter Plaetinck

Post by Jerome Leclanche

Post by Thomas Koch
people reported this issue on different channels. We agreed that it

would

Post by Dieter Plaetinck

Post by Jerome Leclanche

Post by Thomas Koch
make
sense to add a third category STATE which goes into a place of its

own.

Post by Dieter Plaetinck

Post by Jerome Leclanche

Post by Thomas Koch
We don't think that STATE is the same as CACHE. The "backup test" is

not a

Post by Dieter Plaetinck

Post by Jerome Leclanche

Post by Thomas Koch
good indicator. I could live with loosing the state data in an

accident.

Post by Dieter Plaetinck

Post by Jerome Leclanche

Post by Thomas Koch
But I
keep my .cache folder in a tmpfs so that it does not persist across reboots. I
would not like my STATE data to go away with every reboot.

Realistically, that's not feasible. The basedir spec is having a hard
- Plenty of apps still create config files on first run, because that's
just what they do.
- One more dotdir for $HOME, at very little benefit.
- More confusion for developers who want to use the basedir spec. "Does
this count as state?" is a question they now have to ask themselves for
every single setting.
- Most GUI apps use their toolkit's Settings class. State would have to
either be managed separately or be implemented within the toolkit and
specified declaratively somehow. Apart from window geometry, nearly

none of

Post by Dieter Plaetinck

Post by Jerome Leclanche
it would be handled automatically; for the rest it would just be more

work.

Post by Dieter Plaetinck

Post by Jerome Leclanche
-> At this point, you're better off saving window geometry state in

your

Post by Dieter Plaetinck

Post by Jerome Leclanche
WM config.
-> Falling short, there are ways to manage and clean ~/.config
programmatically; that's one of the big bonuses of having this spec.
I've been a big evangelist of the basedir spec and I'm more often than

not

Post by Dieter Plaetinck

Post by Jerome Leclanche
met with developers who (understandably) flat out refuse to split their
config file into a dir, or multiple dirs ("cache is not config", ...).

If

Post by Dieter Plaetinck

Post by Jerome Leclanche
on top of that you start telling them "But this type of config goes

here!",

Post by Dieter Plaetinck

Post by Jerome Leclanche
they'll laugh at you.

I think the xdg basedir spec has good ideals, and sure, it can take some

work to adopt it properly. But this subject comes up often enough to say
it should be dealt with properly.

Post by Dieter Plaetinck
Having a spec that takes care of config, data, cache but not state (at

least not in a proper way) makes the spec incomplete and what's the point
of that?

Post by Dieter Plaetinck
Keeping the spec 3/4-complete as a "favor" to developers who are short on

time/interest seems lame; I would rather finish the spec and if devs don't
feel like implementing it all the way, they should at least accept patches
from people who do.
But who's to say there are only 4 categories? I'm sure we could come up
with 5, 6, or more quite easily. There are all kinds of ways to distinguish
different data. The emphasis should be on what is useful for devs and
users, not on matching some mental model of different kinds of data. And
part of being useful is getting devs to adopt it. An imperfect but widely
used standard beats a perfect standard that half the devs ignore any time.
I'm not saying adding a fourth category is wrong, but I think it should be
justified in terms of practical value and chances of adoption, not just on
theoretical grounds.

I agree, but the proposal for a 4th state category comes up often enough;
i haven't seen anyone advocating a 5th or 6th category.

[Thomas Koch]

Post by Dieter Plaetinck
I think the xdg basedir spec has good ideals, and sure, it can take some
work to adopt it properly. But this subject comes up often enough to say
it should be dealt with properly. Having a spec that takes care of config,
data, cache but not state (at least not in a proper way) makes the spec
incomplete and what's the point of that? Keeping the spec 3/4-complete as
a "favor" to developers who are short on time/interest seems lame; I would
rather finish the spec and if devs don't feel like implementing it all the
way, they should at least accept patches from people who do.

Thank you!

I've added a table that helps to see the difference between STATE and the
other categories:
https://wiki.debian.org/XDGBaseDirectorySpecification

Thomas Koch

[Richard Hartmann]

Dear all,

this discussion has stalled yet again.

Thomas' blog post[1] reminded me that I promised to voice my support
for this idea (matter of fact, I proposed it myself, in the past)
during DebConf13, but failed to do so.

Consider this remedied, now ;)


I can see the argument that this is extra work and not supported, yet.
But the former can be worked around with good documentation and best
practices, the latter is a bit of a fallacy and mainly chicken vs egg.



I would be very happy to help hammer out any such text, assuming
there's a reasonable chance of adoption.


Thanks,
Richard

[1] http://www.koch.ro/blog/index.php?/archives/163-guid.html

[Simon McVittie]

Post by Richard Hartmann
this discussion has stalled yet again.

It has stalled because people aren't convinced that the extra category
is worthwhile.

Looking at the table on
<https://wiki.debian.org/XDGBaseDirectorySpecification>, which I'll
reproduce here in its current state:

DATA | CONFIG | STATE | CACHE | RUNTIME
sync across machines? yes? | yes | no | no | no
manage in VCS? no | yes | no | no | no
should be backed up? yes | yes | yes | no | no
can live in tmpfs? no | no | no | yes | yes?
contains much data? yes | no | no | yes | no

the difference between DATA and STATE appears to be "STATE doesn't
contain much data, and doesn't need to be sync'd across machines". But
surely if it's small, it doesn't matter if it gets sync'd across
machines unnecessarily? If an application wants to store explicitly
per-machine state, it can use the D-Bus/systemd machine ID as a key,
like GNOME does for display settings (I know that's config rather than
state, but the principle is the same).

I agree that storing state in ~/.config is a bug, but I think that's a
misuse of existing categories rather than a need for a new category -
why wouldn't ~/.local/share be OK? - and I don't think adding new
categories is going to reduce developer confusion/misuse between categories.

Other things I'm not sure are right in that table (I'm deliberately not
editing the wiki page without some sort of consensus):

* "CACHE: can live in tmpfs" somewhat defeats the point of a cache:
it's a trade-off between "it might be expensive to download or
regenerate this later" (for a very general definition of
"expensive"), and "I can always download or regenerate it later if I
need to". It *can* go in a tmpfs if statelessness is specifically
considered more valuable than bandwidth/CPU/time, but that shouldn't
happen by default.

* "RUNTIME: can live in tmpfs" should be "yes": the specification for
XDG_RUNTIME_DIR is that it *is* transient. The implementation in
systemd, by the same author as that part of the specification, always
allocates your XDG_RUNTIME_DIR in a tmpfs. I believe Ubuntu has a
reimplementation of XDG_RUNTIME_DIR in the form of a PAM module;
I'm not sure whether it guarantees to use a tmpfs, or uses a fresh
subdirectory of /tmp, or what.

XDG_RUNTIME_DIR is an "odd one out" in that specification: the others
can all be set up correctly by unprivileged application code, but
XDG_RUNTIME_DIR is more like an OS-provided API, and I don't think it
can have its intended semantics without help from the OS.

S

[Ryan Lortie]

hi,

Post by Simon McVittie
I agree that storing state in ~/.config is a bug, but I think that's a
misuse of existing categories rather than a need for a new category -
why wouldn't ~/.local/share be OK? - and I don't think adding new
categories is going to reduce developer confusion/misuse between categories.

Lennart and I had a discussion about this at GUADEC and we came to the
opposite conclusion.

I'll write down here what I got out of that discussion, but I think we
mostly agree.

The message that we want to send is "if in doubt, use ~/.config/".

Specifically, we feel that things should only be "installed" into
~/.local/share/. Things like fonts, plugins, desktop files, dbus
service files, icons, themes, etc. -- essentially things that you would
also find in /usr/share if the sysadmin had installed them instead. I
find this symmetry to be very nice -- and particularly it means that
XDG_DATA_HOME and XDG_DATA_DIRS would then mean the same thing, and I
would be comfortable modifying the spec to be much more strongly-worded
with regards to how you should always use both for the same purpose
(when reading).

I think ~/.config/ is a bit of a weird name for "toss all of the data
here", but so is ~/Library/. ~/.config/ is established. You also have
things like dconf that are storing data here that is definitely
configuration, but also kinda not the sort of "config" you'd expect in
/etc -- ie: you couldn't go in there and edit it with a text editor.
Another example of something that feel a little more state-like but not
exactly config: ~/.config/monitors.xml.

In general, I'd like to tell people that "If your program wants to
install things like plugins|fonts|etc, put them in XDG_DATA_HOME. All
other state goes in XDG_CONFIG_HOME. This includes things like
databases and game state and save files."

We'd also like to tell people that applications putting data in
~/.config/ are expected to use the D-Bus style naming scheme to select a
directory for themselves. We would enforce this for sandboxed
applications. OS components would be exempted from this restriction
(both as a matter of policy and enforcement).

In general, it is expected that you do not place such a subdirectory in
~/.local/share/ -- rather you would install into one of the "well know"
directories for fonts, etc. An exception is if your program has
something like plugins and you have a system directory for that as well
-- then you could name this in the D-Bus style.

The main thrust behind this conclusion is that we've had a lot of people
from a wide range of areas (everyone from game developers to Firefox)
telling us that the current split of "config" vs. "data" is very
difficult for them to support and not something that they have on other
platforms. I agree that adding another directory would not help here.
Blessing "just use ~/.config" as the official way forward would help
these people a lot.

A couple of caveats:

1) I mention "plugins" above while talking about "share". This should
set off alarm bells -- maybe we could also reopen the ~/.lib/x86/ vs.
~/.lib/amd64/ etc. debate (but we should keep that a separate discussion
from this, imho.).

2) ~/.local/share/Trash/ is weird, and thinking about this, would we
honestly expect to have this in ~/.config/? I think that this folder is
truly unique, and putting it in the location that it has now was
probably not a great choice... I think we could just leave it there and
acknowledge that it's weird.

Cheers

[Jerome Leclanche]

Hi,

Post by Ryan Lortie
hi,

Post by Simon McVittie
I agree that storing state in ~/.config is a bug, but I think that's a
misuse of existing categories rather than a need for a new category -
why wouldn't ~/.local/share be OK? - and I don't think adding new
categories is going to reduce developer confusion/misuse between categories.

Lennart and I had a discussion about this at GUADEC and we came to the
opposite conclusion.
I'll write down here what I got out of that discussion, but I think we
mostly agree.
The message that we want to send is "if in doubt, use ~/.config/".
Specifically, we feel that things should only be "installed" into
~/.local/share/. Things like fonts, plugins, desktop files, dbus
service files, icons, themes, etc. -- essentially things that you would
also find in /usr/share if the sysadmin had installed them instead. I
find this symmetry to be very nice -- and particularly it means that
XDG_DATA_HOME and XDG_DATA_DIRS would then mean the same thing, and I
would be comfortable modifying the spec to be much more strongly-worded
with regards to how you should always use both for the same purpose
(when reading).
I think ~/.config/ is a bit of a weird name for "toss all of the data
here", but so is ~/Library/. ~/.config/ is established. You also have
things like dconf that are storing data here that is definitely
configuration, but also kinda not the sort of "config" you'd expect in
/etc -- ie: you couldn't go in there and edit it with a text editor.
Another example of something that feel a little more state-like but not
exactly config: ~/.config/monitors.xml.
In general, I'd like to tell people that "If your program wants to
install things like plugins|fonts|etc, put them in XDG_DATA_HOME. All
other state goes in XDG_CONFIG_HOME. This includes things like
databases and game state and save files."

Very, very strongly agreed with all this for exactly the reasons you
highlighted.

Post by Ryan Lortie
We'd also like to tell people that applications putting data in
~/.config/ are expected to use the D-Bus style naming scheme to select a
directory for themselves. We would enforce this for sandboxed
applications. OS components would be exempted from this restriction
(both as a matter of policy and enforcement).
In general, it is expected that you do not place such a subdirectory in
~/.local/share/ -- rather you would install into one of the "well know"
directories for fonts, etc. An exception is if your program has
something like plugins and you have a system directory for that as well
-- then you could name this in the D-Bus style.

Agreed in general but I'm not sure if entirely dropping the legacy
definition of ~/.local/share is wise.
As long as actual configuration is stored in ~/.config, and actual
cache is stored in ~/.cache, I have no problem with applications using
~/.local/share as their everything-else-dump. Looking at what's in my
/usr/share right now ... it does act as a systemwide "I didn't know
where else to put this" dump. Maybe we can rename the variable to
$XDG_TOSS_IT_WHERE_THERE_IS_ROOM_DIRS. But yes, agreed with the
general idea.
Are you moving towards requiring dbus-style naming? I'm ok with
encouraging it, but not ok with requiring it. Of course, if two apps
conflict, they should move (both of them, and having to handle
migration becomes their own fault for not following the
recommendation).

Post by Ryan Lortie
The main thrust behind this conclusion is that we've had a lot of people
from a wide range of areas (everyone from game developers to Firefox)
telling us that the current split of "config" vs. "data" is very
difficult for them to support and not something that they have on other
platforms. I agree that adding another directory would not help here.
Blessing "just use ~/.config" as the official way forward would help
these people a lot.

This is also the feedback I hear from a lot of people. It's also kinda
what they already do, so the cat is already out of that bag.
People use ~/.config for everything and the apps that don't follow xdg
have a single unsplittable directory. Being able to tell them "go
ahead and just move it to ~/.config" is a lot easier.

Post by Ryan Lortie
1) I mention "plugins" above while talking about "share". This should
set off alarm bells -- maybe we could also reopen the ~/.lib/x86/ vs.
~/.lib/amd64/ etc. debate (but we should keep that a separate discussion
from this, imho.).
2) ~/.local/share/Trash/ is weird, and thinking about this, would we
honestly expect to have this in ~/.config/? I think that this folder is
truly unique, and putting it in the location that it has now was
probably not a great choice... I think we could just leave it there and
acknowledge that it's weird.

It was in ~/.Trash at some point, wasn't it? In hindsight, it was just
fine there and it probably shouldn't have been moved, but it's not
really problematic in ~/.local/share either.

Post by Ryan Lortie
Cheers
_______________________________________________
xdg mailing list
http://lists.freedesktop.org/mailman/listinfo/xdg

[Richard Hartmann]

Post by Jerome Leclanche
Being able to tell them "go
ahead and just move it to ~/.config" is a lot easier.

JFTR, ITYM ~/.config/$0/ .

Post by Jerome Leclanche
It was in ~/.Trash at some point, wasn't it? In hindsight, it was just
fine there and it probably shouldn't have been moved, but it's not
really problematic in ~/.local/share either.

I strongly disagree with upper case letters in mandated file or directory names.

The general clutter of dotdirs could have been reduced by using
~/.xdg/config/$0/ etc by default, but that ship has sailed, as well.


Richard

[Kevin Krammer]

Post by Ryan Lortie
In general, I'd like to tell people that "If your program wants to
install things like plugins|fonts|etc, put them in XDG_DATA_HOME. All
other state goes in XDG_CONFIG_HOME. This includes things like
databases and game state and save files."

I think that cat is out of the bag.
Applications have been using DATA_HOME for data for quite a while now.
If we want to have an INSTALL_HOME [1], we should add that.

Using CONFIG_HOME as the root for everything doesn't sound like a good idea,
IMHO even worse then using DATA_HOME for everything.

Cheers,
Kevin

[1] which would have the added benefit of being something that can be passed
to a user local software build, e.g. configure --prefix $XDG_INSTALL_HOME

[Richard Hartmann]

Post by Kevin Krammer
[1] which would have the added benefit of being something that can be passed
to a user local software build, e.g. configure --prefix $XDG_INSTALL_HOME

Just to play devil's advocate: If that's unset, bad things can happen.
Yes, whatever is parsing --prefix should guard against it, but still.


Richard

[Kevin Krammer]

Post by Richard Hartmann

Post by Kevin Krammer
[1] which would have the added benefit of being something that can be
passed to a user local software build, e.g. configure --prefix
$XDG_INSTALL_HOME

Just to play devil's advocate: If that's unset, bad things can happen.
Yes, whatever is parsing --prefix should guard against it, but still.

Yes, true, I wasn't actually suggesting that literaly :)

Like the other variables there would be a specified default, so configure (or
whatever) could peform the check and fall back to the default or a shell
substitution expression could be used, etc.

I think the usual syntax for meaning "the value of the variable or, if unset
or empty, the default" is to use the name in all lower case.
So confgure --prefix $xdg_install_home :)

Anyway, just wanted to point out that if there is a need for a location that
can host user installed extension instead of data, then the better approach
might be addng a location for that instead of moving data to a new one.

Cheers,
Kevin

[Richard Hartmann]

Post by Kevin Krammer
Like the other variables there would be a specified default, so configure (or
whatever) could peform the check and fall back to the default or a shell
substitution expression could be used, etc.

Those defaults are internal to programs; all a program would see in this case is

./foo --prefix=''

as shell expansion would happen before that. I know what you meant,
but the fact is that you can never solely rely on XDG variables in an
interactive shell.

Post by Kevin Krammer
Anyway, just wanted to point out that if there is a need for a location that
can host user installed extension instead of data, then the better approach
might be addng a location for that instead of moving data to a new one.

Agreed.


Richard

[Kevin Krammer]

Post by Richard Hartmann

Post by Kevin Krammer
Like the other variables there would be a specified default, so configure
(or whatever) could peform the check and fall back to the default or a
shell substitution expression could be used, etc.

Those defaults are internal to programs; all a program would see in this case is
./foo --prefix=''
as shell expansion would happen before that. I know what you meant,
but the fact is that you can never solely rely on XDG variables in an
interactive shell.

Sorry, probably using wrong terminology :)

I meant something like
./foo --prefix=${XDG_INSTALL_HOME:-$HOME/.local}

Cheers,
Kevin

[Richard Hartmann]

Post by Kevin Krammer
I meant something like
./foo --prefix=${XDG_INSTALL_HOME:-$HOME/.local}

That's better indeed :)


Sorry for the tangent, I didn't want to nitpick endlessly, just point
out that people need to take care when using XDG variables.



Richard

[Ted Gould]

Post by Ryan Lortie
We'd also like to tell people that applications putting data in
~/.config/ are expected to use the D-Bus style naming scheme to select a
directory for themselves. We would enforce this for sandboxed
applications. OS components would be exempted from this restriction
(both as a matter of policy and enforcement).
In general, it is expected that you do not place such a subdirectory in
~/.local/share/ -- rather you would install into one of the "well know"
directories for fonts, etc. An exception is if your program has
something like plugins and you have a system directory for that as well
-- then you could name this in the D-Bus style.

What we're doing for Ubuntu application confinement is similar, but
we're allowing for a directory in XDG_DATA_HOME for the application as
well. We don't want an application intermingling of its data into well
known directories in there, and would prefer that they are separate.
We're not using a DBus style name specifically, but very similar, in
that a Click package name is a reverse domain name as well. But, it
doesn't specify the application in the Click package (there can be more
than one). This allows those applications to do things like share a
cache.

Ted

[Simon McVittie]

Post by Ryan Lortie
Specifically, we feel that things should only be "installed" into
~/.local/share/. Things like fonts, plugins, desktop files, dbus
service files, icons, themes, etc. -- essentially things that you would
also find in /usr/share if the sysadmin had installed them instead.

This is a significant policy change. At the moment, ~/.local/share has a
dual role as the home-directory equivalent of both /usr/share and
/var/lib, and is considered to be the right place for user data in
non-document-oriented applications (anything that insta-saves
automatically and doesn't use a Save As dialog), for instance:

* Rhythmbox playlists
* Tomboy notes
* Telepathy (Mission Control) accounts
* Telepathy IM logs
* the Tracker database
* the Evolution address book, calendar etc.

(all of those are non-hypothetical: they exist in my ~/.local/share)

A rule of thumb from

Post by Ryan Lortie
But how do you know what goes in what folder ?
There’s a little trick : just imagine that it was deleted. What would
you think as a user ?
If you would cry, running frenetically to your backup, it means that
it belongs to XDG_DATA_HOME.
If you would think “Damn, I will have to reconfigure all”, it belongs
to $XDG_CONFIG_HOME. [...]
If you would just think “It’s bloody slow those days” then it is
obviously part of XDG_CACHE_HOME.
In general, I'd like to tell people that "If your program wants to
install things like plugins|fonts|etc, put them in XDG_DATA_HOME. All
other state goes in XDG_CONFIG_HOME. This includes things like
databases and game state and save files."

People who put their ~/.config in a VCS are going to hate that change,
and so are people who use deletion of ~/.config as a "factory reset"
(but don't want to lose, e.g., their Tomboy notes).

If the authors of a particular application don't want to distinguish
between configuration and data, that's a feature-request bug that might
reasonably be WONTFIX, but I think XDG_DATA_HOME is a better fallback
for such applications than XDG_CONFIG_HOME.

Post by Ryan Lortie
We'd also like to tell people that applications putting data in
~/.config/ are expected to use the D-Bus style naming scheme to select a
directory for themselves. We would enforce this for sandboxed
applications. OS components would be exempted from this restriction
(both as a matter of policy and enforcement).

I assume service/daemon/behind-the-scenes things like dconf, Telepathy,
GNOME Online Accounts, etc. count as OS components for these purposes?

Post by Ryan Lortie
1) I mention "plugins" above while talking about "share". This should
set off alarm bells -- maybe we could also reopen the ~/.lib/x86/ vs.
~/.lib/amd64/ etc. debate (but we should keep that a separate discussion
from this, imho.).

~/.lib/i386-linux-gnu :-P

Post by Ryan Lortie
2) ~/.local/share/Trash/ is weird

It's weird in a "~/.local/share is the user's /usr/share" world-view. In
a world-view that says "~/.local/share is a hybrid of /usr/share and
/var/lib" it's perfectly normal: if Trash was a system service, it would
be correct for trashed files to go in /var/lib/Trash or something.

S

[Richard Hartmann]

On Wed, Feb 19, 2014 at 4:29 PM, Simon McVittie

Post by Simon McVittie
People who put their ~/.config in a VCS are going to hate that change,
and so are people who use deletion of ~/.config as a "factory reset"
(but don't want to lose, e.g., their Tomboy notes).

We gave up long ago and and resorted to specifying files within
~/.config/$0 as $XDG_CONFIG_HOME is already littered with non-config
bits and pieces; for example state as part of config ;)


Richard

[Thiago Macieira]

Post by Ryan Lortie
1) I mention "plugins" above while talking about "share". This should
set off alarm bells -- maybe we could also reopen the ~/.lib/x86/ vs.
~/.lib/amd64/ etc. debate (but we should keep that a separate discussion
from this, imho.).

~/.local/lib, paralleling /usr/local/lib

If you're building stuff, you could even just do --prefix=$HOME/.local and it
might install correctly.

[Richard Hartmann]

On Wed, Feb 19, 2014 at 1:40 PM, Simon McVittie

Post by Simon McVittie
Looking at the table on
<https://wiki.debian.org/XDGBaseDirectorySpecification>, which I'll

I looked over it myself and wasn't too happy with it.

If it's deemed a requirement to polish the distinctions before
debating the merit of $state (deliberately not implying a storage
location in this context) we can do so, but I think there's a
consensus on this ML what state entails so that may not be needed,
just now.

Post by Simon McVittie
But
surely if it's small, it doesn't matter if it gets sync'd across
machines unnecessarily? If an application wants to store explicitly
per-machine state, it can use the D-Bus/systemd machine ID as a key,
like GNOME does for display settings (I know that's config rather than
state, but the principle is the same).

Thanks for this; it highlights precisely why state is needed.

This is not about storage size, it is about churn in configuration files.

Looking at, e.g., geeqie, there's a section called <layout> in
geeqierc.xml. This contains window size, last path, and a plethora of
stuff configuration management does not care about.

Assuming you put your configuration under proper version control, this
state information changes all the time. It is not something you want
to sync between machines for several reasons. You are synching state
information which may be invalid (lastpath), non-applicable (desktop
resolution settings on a netbook), or outright harmful (this directory
is not needed any more). Additionally, you are filling your VCS
history with random noise, making tracking the actual changes harder.
This results in broken workflows, manual rewriting/pruning of state,
or outright breakage.

And _that_ is why Thomas, me, and others consider carrying state in
configuration such an issue.

Post by Simon McVittie
I agree that storing state in ~/.config is a bug, but I think that's a
misuse of existing categories rather than a need for a new category -
why wouldn't ~/.local/share be OK? - and I don't think adding new
categories is going to reduce developer confusion/misuse between categories.

I would lean towards agreeing with Ryan: ~/.config is well-established
and mis-used already, anyway. Sticking with geeqie, take
.config/geeqie/history . The situation is similar to /etc .
Personally, I would choose $XDG_CONFIG_HOME/$0/state or
$XDG_STATE_HOME/$0/state with $XDG_STATE_HOME defaulting to
$XDG_CONFIG_HOME . It's not pretty, but it would be pragmatic,
fail-safe, and along established "best" practices.

Post by Simon McVittie
it's a trade-off between "it might be expensive to download or
regenerate this later" (for a very general definition of
"expensive"), and "I can always download or regenerate it later if I
need to". It *can* go in a tmpfs if statelessness is specifically
considered more valuable than bandwidth/CPU/time, but that shouldn't
happen by default.

Agreed.

Post by Simon McVittie
* "RUNTIME: can live in tmpfs" should be "yes": the specification for
XDG_RUNTIME_DIR is that it *is* transient. The implementation in
systemd, by the same author as that part of the specification, always
allocates your XDG_RUNTIME_DIR in a tmpfs. I believe Ubuntu has a
reimplementation of XDG_RUNTIME_DIR in the form of a PAM module;
I'm not sure whether it guarantees to use a tmpfs, or uses a fresh
subdirectory of /tmp, or what.

Agreed. Compare /run .

Post by Simon McVittie
XDG_RUNTIME_DIR is an "odd one out" in that specification: the others
can all be set up correctly by unprivileged application code, but
XDG_RUNTIME_DIR is more like an OS-provided API, and I don't think it
can have its intended semantics without help from the OS.

$XDG_OS_RUNTIME_DIR , but that's done and in the past, so...


RIchard

[Kevin Krammer]

Post by Richard Hartmann
On Wed, Feb 19, 2014 at 1:40 PM, Simon McVittie

Post by Simon McVittie
But
surely if it's small, it doesn't matter if it gets sync'd across
machines unnecessarily? If an application wants to store explicitly
per-machine state, it can use the D-Bus/systemd machine ID as a key,
like GNOME does for display settings (I know that's config rather than
state, but the principle is the same).

Thanks for this; it highlights precisely why state is needed.
This is not about storage size, it is about churn in configuration files.
Looking at, e.g., geeqie, there's a section called <layout> in
geeqierc.xml. This contains window size, last path, and a plethora of
stuff configuration management does not care about.
Assuming you put your configuration under proper version control, this
state information changes all the time. It is not something you want
to sync between machines for several reasons. You are synching state
information which may be invalid (lastpath), non-applicable (desktop
resolution settings on a netbook), or outright harmful (this directory
is not needed any more). Additionally, you are filling your VCS
history with random noise, making tracking the actual changes harder.
This results in broken workflows, manual rewriting/pruning of state,
or outright breakage.
And _that_ is why Thomas, me, and others consider carrying state in
configuration such an issue.

So it is mostly about developers using the same file for config and state
instead of using a separete state file?
If it where in a different file, even in the same location, then only the
actual config file would be tracked by the VCS and the state files would not.

Cheers,
Kevin

[Richard Hartmann]

Post by Kevin Krammer
So it is mostly about developers using the same file for config and state
instead of using a separete state file?
If it where in a different file, even in the same location, then only the
actual config file would be tracked by the VCS and the state files would not.

That is what it boils down to, yes.


Richard

[Thomas Koch]

Hi,

I'm happy that the discussion could be revived although the thread drifted a
bit away from the STATE topic.

Maybe it helps to describe a /persona/ for my /user story/:

Axel is an experienced linux power user, working in a team of web developers
and running his own private mail-, file- and web-server. Besides his own
laptop, company laptop, company workstation, several virtual machines he also
administrates the linux laptops of his wife and parents.

Axel uses ZSH, Emacs, tmux, Firefox, Kmail, Awesome Window Manager and Gnome
Classic on Debian. He synchronizes his configuration files with Git (helped by
vcsh) over several machines.

User Story:

Axel wants to be sure, that he tracks all important configuration inside his
$HOME with Git. He does not want to be distracted by files that are not
configuration or that change with every execution of a program or that are
specific to a machine (like window positions, command line history, recently
opened files).

------

It might sound as if the described persona is a minor group of the total linux
user base and could be ignored. On the other hand, once things become possible
for the linux power user, we tend to develop tools that deliver the same
advantages (sync of config) to the rest of the user base.

Post by Simon McVittie
the difference between DATA and STATE appears to be "STATE doesn't
contain much data, and doesn't need to be sync'd across machines". But
surely if it's small, it doesn't matter if it gets sync'd across
machines unnecessarily? If an application wants to store explicitly
per-machine state, it can use the D-Bus/systemd machine ID as a key,
like GNOME does for display settings (I know that's config rather than
state, but the principle is the same).

Well, you're pinpointing it. Syncing window positions or recently opened files
across machines would lead to undesired behaviour because different machines
have different screen setups and differnt sets of files. You put the burden on
the application developer to distinguish between machines, but that's
unrealistic. The xdg-basedir-spec should rather help by providing a directory
for state information that makes sense only in the context of this machine.

Post by Simon McVittie
I agree that storing state in ~/.config is a bug, but I think that's a
misuse of existing categories rather than a need for a new category -
why wouldn't ~/.local/share be OK? - and I don't think adding new
categories is going to reduce developer confusion/misuse between categories.

Using .local/share instead of .config for STATE data would be an improvement.
But since others already raised concerns, it seems that we need a third
category between DATA and CONFIG: STATE.

Post by Simon McVittie
Other things I'm not sure are right in that table (I'm deliberately not
it's a trade-off between "it might be expensive to download or
regenerate this later" (for a very general definition of
"expensive"), and "I can always download or regenerate it later if I
need to". It *can* go in a tmpfs if statelessness is specifically
considered more valuable than bandwidth/CPU/time, but that shouldn't
happen by default.

It's not that important to me whether .cache is in tmpfs or not. But since I
reboot my laptop only once in a month it's ok for me to loose firefox caches,
.thumbnails and a-likes in those cases.

Back to STATE: Yes, it's hard already with the existing categories to get
projects to conform. But I believe that the spec is short on examples and
explanations. This thread and the Debian wiki page already contain some
helpful hints that might be worth to be included in a next version of the
spec.

I also don't have much hope that the long tail of applications will conform to
the spec at any point in time. But I believe that at least libraries will
quickly introduce a STATE directory once it's included in the standard. And
when KDE and Gnome libraries use a separate STATE folder, then we already gain
a lot.

Is there any formal process for the development of the xdg specs? I found the
Git repo and bugtracker today and could open an issue for the STATE DIR
request?

Regards, Thomas Koch

[Kevin Krammer]

Post by Thomas Koch

Post by Simon McVittie
the difference between DATA and STATE appears to be "STATE doesn't
contain much data, and doesn't need to be sync'd across machines". But
surely if it's small, it doesn't matter if it gets sync'd across
machines unnecessarily? If an application wants to store explicitly
per-machine state, it can use the D-Bus/systemd machine ID as a key,
like GNOME does for display settings (I know that's config rather than
state, but the principle is the same).

Well, you're pinpointing it. Syncing window positions or recently opened
files across machines would lead to undesired behaviour because different
machines have different screen setups and differnt sets of files. You put
the burden on the application developer to distinguish between machines,
but that's unrealistic. The xdg-basedir-spec should rather help by
providing a directory for state information that makes sense only in the
context of this machine.

Sorry, slighly offtopic, but:

Window geometry is a rather bad example for a machine specific state, since it
is screen setup specific.

Either application developers really want the burden of handling that
themselves, in which case they need to track screen configurations somehow, or
they do the sensible thing and let the workspace shell/window manager handle
it for them.

My rather limited knowlegde around Wayland even suggests, that the latter will
be the only option there anyway.

Cheers,
Kevin

[Thomas Koch]

Post by Kevin Krammer

Post by Thomas Koch

Post by Simon McVittie
the difference between DATA and STATE appears to be "STATE doesn't
contain much data, and doesn't need to be sync'd across machines". But
surely if it's small, it doesn't matter if it gets sync'd across
machines unnecessarily? If an application wants to store explicitly
per-machine state, it can use the D-Bus/systemd machine ID as a key,
like GNOME does for display settings (I know that's config rather than
state, but the principle is the same).

Well, you're pinpointing it. Syncing window positions or recently opened
files across machines would lead to undesired behaviour because different
machines have different screen setups and differnt sets of files. You put
the burden on the application developer to distinguish between machines,
but that's unrealistic. The xdg-basedir-spec should rather help by
providing a directory for state information that makes sense only in the
context of this machine.

Window geometry is a rather bad example for a machine specific state,

since

Post by Kevin Krammer
it is screen setup specific.
Either application developers really want the burden of handling that
themselves, in which case they need to track screen configurations

somehow,

Post by Kevin Krammer
or they do the sensible thing and let the workspace shell/window manager
handle it for them.
My rather limited knowlegde around Wayland even suggests, that the latter
will be the only option there anyway.
Cheers,
Kevin

KDE is my posterchild example for saving window and a lot of other state in a
config file. There is also some useful data in kmailrc that I'd like to keep
when setting up a new machine. But most of the data is state:

cat ~/.kde/share/config/kmailrc | grep -A 5 Window
ReaderWindowHeight=1334
SearchAndHeaderHeight=388
SearchAndHeaderWidth=1334
composer=523,1624

[HTML Settings]
--
[Main Window]
Height 1050=1048
Height 1600=500
Height 1680=1624
State=AAAA/wAAAAD9AAAAAAAABo4AAAO5AAAABAAAAAQAAAAIAAAACPwAAAABAAAAAgAAAAEAAAAWAG0AYQBpAG4AVABvAG8AbABCAGEAcgEAAAAA/////wAAAAAAAAAA
ToolBarsMovable=Disabled
--
[Separate Reader Window]
Height 1050=1048
Height 1680=1624
State=AAAA/wAAAAD9AAAAAAAAAgsAAAX7AAAABAAAAAQAAAAIAAAACPwAAAABAAAAAgAAAAEAAAAWAG0AYQBpAG4AVABvAG8AbABCAGEAcgEAAAAA/////wAAAAAAAAAA
ToolBarsMovable=Disabled
Width 1050=523


Regards, Thomas Koch

[Richard Hartmann]

Post by Thomas Koch
Back to STATE: Yes, it's hard already with the existing categories to get
projects to conform. But I believe that the spec is short on examples and
explanations. This thread and the Debian wiki page already contain some
helpful hints that might be worth to be included in a next version of the
spec.

A good spec is done when there's nothing left to remove, any more.

That being said, I tried to get some changes into that document
several years ago as it's needlessly complicated and could use an
appendix, imo. I suspect that familiarity with the specs make it hard
for XDG veterans to see it like relative newcomers. This is not meant
as an offense, this just happens, sometimes.

Long story short, I think the specs could use a rewrite, no matter
what happens with state.

I would gladly help out if given some guidance on what to keep in mind
and if this would be something that's likely to be merged.


Thanks,
Richard