Sunday, March 18, 2018

[389-users] Re: autosizing the cache

On Tue, 2018-03-13 at 21:10 -0500, Sergei Gerasenko wrote:
> Hi William,
> With autosizing on I configured the changelog backend:
> dn: cn=changelog,cn=ldbm database,cn=plugins,cn=config
> cn: changelog
> objectClass: top
> objectClass: extensibleObject
> objectClass: nsBackendInstance
> nsslapd-suffix: cn=changelog
> nsslapd-cachesize: -1
> nsslapd-cachememsize: 8858370048
> nsslapd-readonly: off
> nsslapd-require-index: off
> nsslapd-directory: /var/lib/dirsrv/slapd-CNVR-NET/db/changelog
> nsslapd-dncachememsize: 3000000000
> Here's the ldif:
> dn: cn=changelog,cn=ldbm database,cn=plugins,cn=config
> changetype: modify
> replace: nsslapd-dncachememsize
> nsslapd-dncachememsize: 3000000000
> -
> replace: nsslapd-cachememsize
> nsslapd-cachememsize: 3000000000
> The server refused to change nsslapd-cachememsize because of
> autosizing but nsslapd-dncachememsize did get changed. So it seems
> that I can still control nsslapd-dncachememsize? When I restart 389-
> ds, the 3G persists as well.
> Does that make sense?

Yes it does.

> Thank you for a quick response!

At the current point in time, dncachememsize is NOT controlled by
autosizing. There is some ideas in my head about how to improve this,
but the issue is legacy. We have one control: autosize percentage. But
this doesn't really represent a complete picture and story about how we
size our backends and data.

We have to continue to support the current variables, so I can't easily
change this from it's current form.

*IF* I was given unlimited power (I never will be ;) ) I would probably
make the interface something like:

autosize-max-memory: <value in bytes/mb/gb>
entrycache-slice: A%
dncache-slice: B%
dbcache-slice: C%
ndncache-slice: D%

So how would this work?

You have autosize max that is the "total ram" we want to use. Lets say
2GB. Each backend would be given it's own "limit". We'd try to detect
some sane values on your system of course, but we can only do so much
:) So say ... each backend gets 5-10% of system ram limit. (We have to
be super conservative due to dogtag in freeipa which eats ram).

Then we slice that up. So entrycache-slice + dncache-slice + dbcache-
slice + ndncache-slice = 100%. So by default we might do something

entrycache-slice: 75
dbcache-slice: 10
dncache-slice: 10
ndncache-slice: 5

So this on a 1GB backend translates to:

750MB of entrycache
100MB of dncache
100MB of dbcache
50MB of ndncache

of course some tests would be needed to find the right slice values by

Then it's as simple as "if you want more from your backend" just up the
one memlimit value and "everything" increases in a correct proportional
rate. Rather than thinking about how to hand tune everything, we make
informed design decisions, you just say "here's mem max". You have
control to say each backend gets a different value (say FreeIPA you may
want 4GB to userRoot, 1GB to CA system backend). you still have lots of
flexibility as an admin, but you don't have to worry about so many
moving interacting parts.

Hope that helps,

> Sergei
> > On Mar 13, 2018, at 7:40 PM, William Brown <
> > u> wrote:
> >
> > If autosize > 0, we write a new entrychace/cachememsize every start
> > up.
> >
> > So you only need to set autosize to between 1 and 99 for it to
> > work.
> >
> > There is some other logic in there to account for other scenarioes
> > =
> > for example, if you set autosize to 0 AND you set the entry
> > cachesize
> > to 0, we'll autosize it at start up anyway. BUT if you have
> > autosize =
> > 0, and entry cachesize > 0, we won't touch it.
> _______________________________________________
> 389-users mailing list --
> To unsubscribe send an email to 389-users-leave@lists.fedoraproject.o
> rg

William Brown
389-users mailing list --
To unsubscribe send an email to

No comments:

Post a Comment