Thursday, November 26, 2015

[389-users] Re: Slow search results until cache populated

-------------------------------------------------
cn=monitor,cn=ldbm database,cn=plugins,cn=config
-------------------------------------------------
dn: cn=monitor,cn=ldbm database,cn=plugins,cn=config
objectClass: top
objectClass: extensibleObject
cn: monitor
database: ldbm database
dbcachehits: 27519011
dbcachetries: 27551904
dbcachehitratio: 99
dbcachepagein: 32893
dbcachepageout: 18366
dbcacheroevict: 10852
dbcacherwevict: 1804

-------------------------------------------------
cn=userRoot,cn=ldbm database,cn=plugins,cn=config
-------------------------------------------------
dn: cn=userRoot,cn=ldbm database,cn=plugins,cn=config
objectClass: top
objectClass: extensibleObject
objectClass: nsBackendInstance
cn: userRoot
nsslapd-suffix: o=cctpu
nsslapd-cachesize: -1
nsslapd-cachememsize: 1073741824
nsslapd-readonly: off
nsslapd-require-index: off
nsslapd-directory: /var/lib/dirsrv/slapd-ldap90/db/userRoot
nsslapd-dncachememsize: 4294966

-------------------------------------------------
cn=monitor,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
-------------------------------------------------
dn: cn=monitor,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
objectClass: top
objectClass: extensibleObject
cn: monitor
database: ldbm database
readonly: 0
entrycachehits: 1931
entrycachetries: 2901
entrycachehitratio: 66
currententrycachesize: 3492014
maxentrycachesize: 1073741824
currententrycachecount: 950
maxentrycachecount: -1
dncachehits: 948
dncachetries: 68296
dncachehitratio: 1
currentdncachesize: 4294937
maxdncachesize: 4294966
currentdncachecount: 25280
maxdncachecount: -1
normalizeddncachetries: 473382
normalizeddncachehits: 318441
normalizeddncachemisses: 154941
normalizeddncachehitratio: 67
currentnormalizeddncachesize: 20235761
maxnormalizeddncachesize: 20971520
currentnormalizeddncachecount: 84940
dbfilename-0: userRoot/tpuMemberOf.db
dbfilecachehit-0: 10235
dbfilecachemiss-0: 2
dbfilepagein-0: 2
dbfilepageout-0: 17
dbfilename-1: userRoot/mobile.db
dbfilecachehit-1: 1
dbfilecachemiss-1: 0
dbfilepagein-1: 0
dbfilepageout-1: 0
dbfilename-2: userRoot/entryrdn.db
dbfilecachehit-2: 1134865
dbfilecachemiss-2: 867
dbfilepagein-2: 867
dbfilepageout-2: 0
dbfilename-3: userRoot/ou.db
dbfilecachehit-3: 286
dbfilecachemiss-3: 0
dbfilepagein-3: 0
dbfilepageout-3: 0
dbfilename-4: userRoot/professionallyinterestedin.db
dbfilecachehit-4: 11360
dbfilecachemiss-4: 2
dbfilepagein-4: 2
dbfilepageout-4: 17
dbfilename-5: userRoot/tpuattendscourse.db
dbfilecachehit-5: 140662
dbfilecachemiss-5: 5
dbfilepagein-5: 5
dbfilepageout-5: 267
dbfilename-6: userRoot/tamkspecialization.db
dbfilecachehit-6: 19599
dbfilecachemiss-6: 11
dbfilepagein-6: 11
dbfilepageout-6: 34
dbfilename-7: userRoot/tpuSignsOnForCourse.db
dbfilecachehit-7: 0
dbfilecachemiss-7: 1
dbfilepagein-7: 1
dbfilepageout-7: 0
dbfilename-8: userRoot/hobby.db
dbfilecachehit-8: 0
dbfilecachemiss-8: 1
dbfilepagein-8: 1
dbfilepageout-8: 0
dbfilename-9: userRoot/cosspecifier.db
dbfilecachehit-9: 0
dbfilecachemiss-9: 1
dbfilepagein-9: 1
dbfilepageout-9: 0
dbfilename-11: userRoot/aci.db
dbfilecachehit-11: 3
dbfilecachemiss-11: 1
dbfilepagein-11: 1
dbfilepageout-11: 1
dbfilename-12: userRoot/mail.db
dbfilecachehit-12: 77
dbfilecachemiss-12: 0
dbfilepagein-12: 0
dbfilepageout-12: 0
dbfilename-13: userRoot/title.db
dbfilecachehit-13: 104
dbfilecachemiss-13: 0
dbfilepagein-13: 0
dbfilepageout-13: 0
dbfilename-14: userRoot/tpuBusinessCategoryNumber.db
dbfilecachehit-14: 3075
dbfilecachemiss-14: 2
dbfilepagein-14: 2
dbfilepageout-14: 6
dbfilename-15: userRoot/wid.db
dbfilecachehit-15: 702493
dbfilecachemiss-15: 40
dbfilepagein-15: 40
dbfilepageout-15: 752
dbfilename-16: userRoot/tpuCourseTeacher.db
dbfilecachehit-16: 128921
dbfilecachemiss-16: 4
dbfilepagein-16: 4
dbfilepageout-16: 173
dbfilename-17: userRoot/tpuADGroupId.db
dbfilecachehit-17: 553
dbfilecachemiss-17: 2
dbfilepagein-17: 2
dbfilepageout-17: 5
dbfilename-18: userRoot/tpuStudyOption.db
dbfilecachehit-18: 8776
dbfilecachemiss-18: 2
dbfilepagein-18: 2
dbfilepageout-18: 17
dbfilename-19: userRoot/cn.db
dbfilecachehit-19: 6666863
dbfilecachemiss-19: 93
dbfilepagein-19: 93
dbfilepageout-19: 4369
dbfilename-20: userRoot/nsRoleDN.db
dbfilecachehit-20: 0
dbfilecachemiss-20: 1
dbfilepagein-20: 1
dbfilepageout-20: 0
dbfilename-21: userRoot/tpuSSID.db
dbfilecachehit-21: 1
dbfilecachemiss-21: 0
dbfilepagein-21: 0
dbfilepageout-21: 0
dbfilename-22: userRoot/parentid.db
dbfilecachehit-22: 7
dbfilecachemiss-22: 0
dbfilepagein-22: 0
dbfilepageout-22: 0
dbfilename-25: userRoot/tpuCourseHomeUnit.db
dbfilecachehit-25: 202414
dbfilecachemiss-25: 2
dbfilepagein-25: 2
dbfilepageout-25: 80
dbfilename-26: userRoot/memberUid.db
dbfilecachehit-26: 45
dbfilecachemiss-26: 2
dbfilepagein-26: 2
dbfilepageout-26: 1
dbfilename-27: userRoot/toisuRole.db
dbfilecachehit-27: 129
dbfilecachemiss-27: 2
dbfilepagein-27: 2
dbfilepageout-27: 1
dbfilename-28: userRoot/tpuOrgLevel3.db
dbfilecachehit-28: 5044
dbfilecachemiss-28: 2
dbfilepagein-28: 2
dbfilepageout-28: 12
dbfilename-30: userRoot/tpuConfluGroup.db
dbfilecachehit-30: 171387
dbfilecachemiss-30: 3
dbfilepagein-30: 3
dbfilepageout-30: 86
dbfilename-31: userRoot/department.db
dbfilecachehit-31: 20
dbfilecachemiss-31: 0
dbfilepagein-31: 0
dbfilepageout-31: 0
dbfilename-32: userRoot/tpuCourseImplementation.db
dbfilecachehit-32: 180540
dbfilecachemiss-32: 2
dbfilepagein-32: 2
dbfilepageout-32: 207
dbfilename-33: userRoot/tpuNoFeedBackCourse.db
dbfilecachehit-33: 7
dbfilecachemiss-33: 2
dbfilepagein-33: 2
dbfilepageout-33: 1
dbfilename-35: userRoot/tpuDegree.db
dbfilecachehit-35: 8127
dbfilecachemiss-35: 5
dbfilepagein-35: 5
dbfilepageout-35: 16
dbfilename-36: userRoot/tpuContactSource.db
dbfilecachehit-36: 22867
dbfilecachemiss-36: 2
dbfilepagein-36: 2
dbfilepageout-36: 19
dbfilename-37: userRoot/uid.db
dbfilecachehit-37: 59666
dbfilecachemiss-37: 6
dbfilepagein-37: 6
dbfilepageout-37: 149
dbfilename-38: userRoot/tpuQprIsChief.db
dbfilecachehit-38: 0
dbfilecachemiss-38: 1
dbfilepagein-38: 1
dbfilepageout-38: 0
dbfilename-39: userRoot/mailEquivalentAddress.db
dbfilecachehit-39: 1
dbfilecachemiss-39: 0
dbfilepagein-39: 0
dbfilepageout-39: 0
dbfilename-41: userRoot/memberOf.db
dbfilecachehit-41: 0
dbfilecachemiss-41: 2
dbfilepagein-41: 2
dbfilepageout-41: 1
dbfilename-42: userRoot/tpuOrgLevel1.db
dbfilecachehit-42: 6429
dbfilecachemiss-42: 2
dbfilepagein-42: 2
dbfilepageout-42: 7
dbfilename-43: userRoot/uidNumber.db
dbfilecachehit-43: 79686
dbfilecachemiss-43: 2
dbfilepagein-43: 2
dbfilepageout-43: 94
dbfilename-44: userRoot/mailHost.db
dbfilecachehit-44: 2
dbfilecachemiss-44: 2
dbfilepagein-44: 2
dbfilepageout-44: 1
dbfilename-46: userRoot/winhainigroup.db
dbfilecachehit-46: 24547
dbfilecachemiss-46: 2
dbfilepagein-46: 2
dbfilepageout-46: 48
dbfilename-50: userRoot/tpuDegreeMajorSubject.db
dbfilecachehit-50: 10990
dbfilecachemiss-50: 12
dbfilepagein-50: 12
dbfilepageout-50: 26
dbfilename-53: userRoot/tpuNativeLanguage.db
dbfilecachehit-53: 1
dbfilecachemiss-53: 2
dbfilepagein-53: 2
dbfilepageout-53: 1
dbfilename-54: userRoot/tpuGoogleId.db
dbfilecachehit-54: 5497
dbfilecachemiss-54: 2
dbfilepagein-54: 2
dbfilepageout-54: 14
dbfilename-56: userRoot/givenName.db
dbfilecachehit-56: 1
dbfilecachemiss-56: 0
dbfilepagein-56: 0
dbfilepageout-56: 0
dbfilename-57: userRoot/gidNumber.db
dbfilecachehit-57: 1
dbfilecachemiss-57: 0
dbfilepagein-57: 0
dbfilepageout-57: 0
dbfilename-58: userRoot/tpuMigrSchedule.db
dbfilecachehit-58: 1
dbfilecachemiss-58: 0
dbfilepagein-58: 0
dbfilepageout-58: 0
dbfilename-60: userRoot/tpuStudyProgram.db
dbfilecachehit-60: 1871
dbfilecachemiss-60: 2
dbfilepagein-60: 2
dbfilepageout-60: 4
dbfilename-61: userRoot/tpuCourseImplementationNumber.db
dbfilecachehit-61: 203464
dbfilecachemiss-61: 2
dbfilepagein-61: 2
dbfilepageout-61: 124
dbfilename-62: userRoot/uniquemember.db
dbfilecachehit-62: 151487
dbfilecachemiss-62: 2
dbfilepagein-62: 2
dbfilepageout-62: 424
dbfilename-63: userRoot/mailAlternateAddress.db
dbfilecachehit-63: 46372
dbfilecachemiss-63: 2
dbfilepagein-63: 2
dbfilepageout-63: 163
dbfilename-65: userRoot/nsTombstoneCSN.db
dbfilecachehit-65: 22
dbfilecachemiss-65: 3
dbfilepagein-65: 3
dbfilepageout-65: 2
dbfilename-66: userRoot/o.db
dbfilecachehit-66: 20
dbfilecachemiss-66: 3
dbfilepagein-66: 3
dbfilepageout-66: 2
dbfilename-68: userRoot/tpuCourseImplementationStarts.db
dbfilecachehit-68: 967487
dbfilecachemiss-68: 2
dbfilepagein-68: 2
dbfilepageout-68: 497
dbfilename-69: userRoot/tpuJobDescription.db
dbfilecachehit-69: 28365
dbfilecachemiss-69: 4
dbfilepagein-69: 4
dbfilepageout-69: 39
dbfilename-71: userRoot/telephoneNumber.db
dbfilecachehit-71: 1
dbfilecachemiss-71: 0
dbfilepagein-71: 0
dbfilepageout-71: 0
dbfilename-72: userRoot/tpuToolRole.db
dbfilecachehit-72: 69
dbfilecachemiss-72: 2
dbfilepagein-72: 2
dbfilepageout-72: 1
dbfilename-73: userRoot/tpuSpokenLanguage.db
dbfilecachehit-73: 7588
dbfilecachemiss-73: 2
dbfilepagein-73: 2
dbfilepageout-73: 13
dbfilename-75: userRoot/numsubordinates.db
dbfilecachehit-75: 0
dbfilecachemiss-75: 0
dbfilepagein-75: 0
dbfilepageout-75: 0
dbfilename-76: userRoot/tpuCourseCode.db
dbfilecachehit-76: 721465
dbfilecachemiss-76: 2
dbfilepagein-76: 2
dbfilepageout-76: 733
dbfilename-77: userRoot/tpuCourseImplementationEnds.db
dbfilecachehit-77: 951011
dbfilecachemiss-77: 2
dbfilepagein-77: 2
dbfilepageout-77: 515
dbfilename-78: userRoot/owner.db
dbfilecachehit-78: 1
dbfilecachemiss-78: 2
dbfilepagein-78: 2
dbfilepageout-78: 1
dbfilename-79: userRoot/groupid.db
dbfilecachehit-79: 16615
dbfilecachemiss-79: 2
dbfilepagein-79: 2
dbfilepageout-79: 28
dbfilename-80: userRoot/mailUserStatus.db
dbfilecachehit-80: 22597
dbfilecachemiss-80: 2
dbfilepagein-80: 2
dbfilepageout-80: 16
dbfilename-81: userRoot/tpuSympaHRId.db
dbfilecachehit-81: 8191
dbfilecachemiss-81: 2
dbfilepagein-81: 2
dbfilepageout-81: 12
dbfilename-82: userRoot/sn.db
dbfilecachehit-82: 781941
dbfilecachemiss-82: 2
dbfilepagein-82: 2
dbfilepageout-82: 779
dbfilename-83: userRoot/departmentNumber.db
dbfilecachehit-83: 3348
dbfilecachemiss-83: 2
dbfilepagein-83: 2
dbfilepageout-83: 8
dbfilename-84: userRoot/objectclass.db
dbfilecachehit-84: 437
dbfilecachemiss-84: 0
dbfilepagein-84: 0
dbfilepageout-84: 0
dbfilename-85: userRoot/nsuniqueid.db
dbfilecachehit-85: 29
dbfilecachemiss-85: 4
dbfilepagein-85: 4
dbfilepageout-85: 0
dbfilename-86: userRoot/tpuMigrPSTMainStatus.db
dbfilecachehit-86: 254491
dbfilecachemiss-86: 95
dbfilepagein-86: 95
dbfilepageout-86: 365
dbfilename-87: userRoot/tpuOrgCompositePrimary.db
dbfilecachehit-87: 156765
dbfilecachemiss-87: 54
dbfilepagein-87: 54
dbfilepageout-87: 197
dbfilename-92: userRoot/id2entry.db
dbfilecachehit-92: 70366
dbfilecachemiss-92: 31484
dbfilepagein-92: 31484
dbfilepageout-92: 10
dbfilename-93: userRoot/tpuMainStudyProgram.db
dbfilecachehit-93: 1480064
dbfilecachemiss-93: 13
dbfilepagein-93: 13
dbfilepageout-93: 1013
dbfilename-95: userRoot/iplanet-am-modifiable-by.db
dbfilecachehit-95: 41519
dbfilecachemiss-95: 2
dbfilepagein-95: 2
dbfilepageout-95: 21
dbfilename-96: userRoot/winhagroup.db
dbfilecachehit-96: 0
dbfilecachemiss-96: 1
dbfilepagein-96: 1
dbfilepageout-96: 0
dbfilename-97: userRoot/tpuMigrMainStatus.db
dbfilecachehit-97: 1
dbfilecachemiss-97: 0
dbfilepagein-97: 0
dbfilepageout-97: 0
dbfilename-99: userRoot/tpuOrgCompositeList.db
dbfilecachehit-99: 692
dbfilecachemiss-99: 2
dbfilepagein-99: 2
dbfilepageout-99: 1
dbfilename-101: userRoot/street.db
dbfilecachehit-101: 32844
dbfilecachemiss-101: 2
dbfilepagein-101: 2
dbfilepageout-101: 48
dbfilename-112: userRoot/tpuOrgLevel2.db
dbfilecachehit-112: 8325
dbfilecachemiss-112: 2
dbfilepagein-112: 2
dbfilepageout-112: 16
dbfilename-113: userRoot/tpuGender.db
dbfilecachehit-113: 35145
dbfilecachemiss-113: 2
dbfilepagein-113: 2
dbfilepageout-113: 20
dbfilename-120: userRoot/ancestorid.db
dbfilecachehit-120: 526
dbfilecachemiss-120: 0
dbfilepagein-120: 0
dbfilepageout-120: 0


Hi,

Thank you for your answer!
Here the result of the rpm command:
389-ds-base-1.3.3.1-23.el7_1.x86_64
389-ds-base-1.3.4.0-19.el7.x86_64
389-ds-base-libs-1.3.4.0-19.el7.x86_64
389-ds-base-devel-1.3.4.0-19.el7.x86_64

The ldapsearch results are enclosed. You have to take into account that
this is not yet in production - will be very soon and that's why I'm a
little bit concerned about the performance :)

I was just thinking how well indexing works at all for searches that
contain nsrole-attribute conditions, since nsrole is a dynamic computed
attribute - we are using roles a lot and searches may include logical
combinations of many roles.

Regards,
Petteri

On 25.11.2015 23:20, William Brown wrote:
> On Wed, 2015-11-25 at 09:35 +0200, Petteri Jekunen wrote:
>> Hi,
>>
>> Is it just ordinary behavior with 389 DS that search results may take
>> a
>> very loong time just after starting the server when there are no
>> entries
>> in the cache yet?
>> And when the cache is fully saturated (enough cache configured for
>> all
>> the entries) results become dramatically down - for instance from 4
>> minutes to 4 seconds.
>>
>> If this this is so, is there anything that could be done to fill in
>> the
>> cache automatically after startup?
>>
>> We have some 60 000 entries, RHEL 7.1, 389-Directory/1.3.3.1
>> B2015.267.1737 on VMWare.
>> We have quite a heavy use of roles, and this seems to be a
>> significant
>> issue especially with them - or at least with them.
>> We have used the Sun DSEE previously and are quite new to 389 DS. The
>> technology seems very similar although.
>>
>> Thanks a lot in advance!
>
>
> Can you post:
>
> rpm -qa | grep 389-ds-base
> for the rpm version
>
> As well, can we see:
>
> ldapsearch -b cn=monitor,cn=ldbm database,cn=plugins,cn=config -s base
> ldapsearch -b cn=userRoot,cn=ldbm database,cn=plugins,cn=config -s base
> ldapsearch -b cn=monitor,cn=userRoot,cn=ldbm
> database,cn=plugins,cn=config -s base
>
> This will show what the cache hit rates and sizing are.
>
> You may find that the issue is a lack of key indexes, and that once the
> cache is primed that is masking the issue. Perhaps look in the access
> log for notes=U ?
>
>
>
> --
> 389 users mailing list
> 389-users@%(host_name)s
> http://lists.fedoraproject.org/admin/lists/389-users@lists.fedoraproject.org
>

No comments:

Post a Comment