Wednesday, July 17, 2024

[389-users] Re: Replication weirdness with 3.0.1 mdb instance

FYI: Indeed it is a bug.
Yesterday I created https://github.com/389ds/389-ds-base/issues/6265 to track it and we are working on a fix.
The message may happen or not (depending whether the data have been added through ldap operation or through an import) 

Regards
    Pierre

On Wed, Jul 17, 2024 at 8:05 AM Zechmeister Christopher <christopher.zechmeister@apa.at> wrote:
Good Morning,

I'm having a similar setup - MMR with only 2 backends, both running mdb - and the same problem with online init of replication, suddenly stopping to replicate my ~1300 users at about 520 - slightly varying as far as I remember. And I can confirm, that I saw exactly your message on supplier side. Unfortunately I already removed the instance and switched back to bdb, so I lost my logs...

Best regards
Christopher

On 03.07.2024, at 16:22, Pierre Rogier <progier@redhat.com> wrote:

Einige Personen, die diese Nachricht erhalten haben, erhalten nicht oft eine E-Mail von progier@redhat.com. Erfahren Sie, warum dies wichtig ist
Hi,
I suspect it may be a bug that we are just discovering:
Have you checked the error on the supplier  you are initializing from ?

Are you seeing something like:
[03/Jul/2024:16:07:57.719471684 +0200] - ERR - check_suffix_entryID - Unable to retrieve entryid of the suffix entry dc=example,dc=com



On Tue, Jul 2, 2024 at 8:33 PM <tdarby@arizona.edu> wrote:
I have several 389ds instances using MMR, but only one at the moment that uses 3.0.1 and the new mdb. It's an instance I'm building from scratch and I script it with config files that are similar to the 2.5 instances. When I initiate replication, what I see is that it finishes without errors but only sends approx. 1000 entries to the consumer (there are over 1M entries). I've looked at all the config attributes that I'm aware of and I'm stumped.
Can you think of anything that would prevent a replication account from sending over more than 1000 entries? Here's a typical snippet from the logs for this:

[02/Jul/2024:17:22:05.137366809 +0000] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica ou=netid,ou=ccit,o=university of arizona,c=us is going offline; disabling replication
[02/Jul/2024:17:22:07.913831452 +0000] - INFO - dbmdb_import_monitor_threads - import dsroot: Import writer thread usage: run: 9.31% read: 69.38% write: 19.17% pause: 0.99% txnbegin: 0.00% txncommit: 1.15%
[02/Jul/2024:17:22:07.978516129 +0000] - INFO - dbmdb_import_monitor_threads - import dsroot: Workers finished; cleaning up...
[02/Jul/2024:17:22:07.981201324 +0000] - INFO - dbmdb_import_monitor_threads - import dsroot: Workers cleaned up.
[02/Jul/2024:17:22:07.983442852 +0000] - INFO - dbmdb_public_dbmdb_import_main - import dsroot: Indexing complete.  Post-processing...
[02/Jul/2024:17:22:07.985709898 +0000] - INFO - dbmdb_public_dbmdb_import_main - import dsroot: Generating numsubordinates (this may take several minutes to complete)...
[02/Jul/2024:17:22:07.991134437 +0000] - INFO - dbmdb_public_dbmdb_import_main - import dsroot: Generating numSubordinates complete.
[02/Jul/2024:17:22:07.993601582 +0000] - INFO - dbmdb_public_dbmdb_import_main - import dsroot: Flushing caches...
[02/Jul/2024:17:22:07.995667599 +0000] - INFO - dbmdb_public_dbmdb_import_main - import dsroot: Closing files...
[02/Jul/2024:17:22:07.997941832 +0000] - INFO - dbmdb_public_dbmdb_import_main - import dsroot: Import complete.  Processed 1097 entries in 3 seconds. (365.67 entries/sec)
--
_______________________________________________
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue


--
--

389 Directory Server Development Team
--
_______________________________________________
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

--
_______________________________________________
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue


--
--

389 Directory Server Development Team

[389-users] Re: help with replication changelog size

Hi,

Some other possibilities to complete Mark's answer:
 replication misconfiguration on the consumer (replication role configured as hub or as supplier instead of consumer) 
 if it is about the changelog db (i.e the retro changelog databases) and in this case it is the retro changelog trimming configuration that are missing. 

FYI: there is a known issue about replication_changelog.db is getting wrongly created on consumer but the db file should not grow but stay nearly empty  

Regards
  Pierre

On Tue, Jul 16, 2024 at 7:32 PM Mark Reynolds <mareynol@redhat.com> wrote:
Changed the subject of this email as this message was filtered into my
junk folder...

On 7/15/24 1:05 PM, kamakshya nayak wrote:
> I have a cascading multi setup 389-ds.
> I observed that on my supplier, hub and consumer are doing good in
> terms of replication.
> They are healthy.
> What I observe is supplier and hub is doing good with replication and
> size of changelog.db for one of the backend suffix is less.
> But, consumer changelog.db disk space utilization is more around 34 Gb
> and supplier and hub changelog.db disk space utilization  is 40 mb.
> What could be the issue and why it's utilizing that high disk space
> for changelog.db ?

What version of 389-ds-base are you using?

At first glance it would appear you have changelog trimming configured
on the supplier/hub but not on the consumer.  That could be the issue,
or it could be changelog compaction.  On the other hand a consumer does
not need a changelog so you could remove it's configuration.

Also I don't see a "changelog.db" file in my setup, so maybe you are on
a very old version?

Mark

>
> And, why a consumer is holding changelog.db file.. where could be the
> issue.
>
> Thank you.
>
--
Identity Management Development Team

--
_______________________________________________
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue


--
--

389 Directory Server Development Team

Tuesday, July 16, 2024

[389-users] Re: Replication weirdness with 3.0.1 mdb instance

Good Morning,

I'm having a similar setup - MMR with only 2 backends, both running mdb - and the same problem with online init of replication, suddenly stopping to replicate my ~1300 users at about 520 - slightly varying as far as I remember. And I can confirm, that I saw exactly your message on supplier side. Unfortunately I already removed the instance and switched back to bdb, so I lost my logs...

Best regards
Christopher

On 03.07.2024, at 16:22, Pierre Rogier <progier@redhat.com> wrote:

Einige Personen, die diese Nachricht erhalten haben, erhalten nicht oft eine E-Mail von progier@redhat.com. Erfahren Sie, warum dies wichtig ist
Hi,
I suspect it may be a bug that we are just discovering:
Have you checked the error on the supplier  you are initializing from ?

Are you seeing something like:
[03/Jul/2024:16:07:57.719471684 +0200] - ERR - check_suffix_entryID - Unable to retrieve entryid of the suffix entry dc=example,dc=com



On Tue, Jul 2, 2024 at 8:33 PM <tdarby@arizona.edu> wrote:
I have several 389ds instances using MMR, but only one at the moment that uses 3.0.1 and the new mdb. It's an instance I'm building from scratch and I script it with config files that are similar to the 2.5 instances. When I initiate replication, what I see is that it finishes without errors but only sends approx. 1000 entries to the consumer (there are over 1M entries). I've looked at all the config attributes that I'm aware of and I'm stumped.
Can you think of anything that would prevent a replication account from sending over more than 1000 entries? Here's a typical snippet from the logs for this:

[02/Jul/2024:17:22:05.137366809 +0000] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica ou=netid,ou=ccit,o=university of arizona,c=us is going offline; disabling replication
[02/Jul/2024:17:22:07.913831452 +0000] - INFO - dbmdb_import_monitor_threads - import dsroot: Import writer thread usage: run: 9.31% read: 69.38% write: 19.17% pause: 0.99% txnbegin: 0.00% txncommit: 1.15%
[02/Jul/2024:17:22:07.978516129 +0000] - INFO - dbmdb_import_monitor_threads - import dsroot: Workers finished; cleaning up...
[02/Jul/2024:17:22:07.981201324 +0000] - INFO - dbmdb_import_monitor_threads - import dsroot: Workers cleaned up.
[02/Jul/2024:17:22:07.983442852 +0000] - INFO - dbmdb_public_dbmdb_import_main - import dsroot: Indexing complete.  Post-processing...
[02/Jul/2024:17:22:07.985709898 +0000] - INFO - dbmdb_public_dbmdb_import_main - import dsroot: Generating numsubordinates (this may take several minutes to complete)...
[02/Jul/2024:17:22:07.991134437 +0000] - INFO - dbmdb_public_dbmdb_import_main - import dsroot: Generating numSubordinates complete.
[02/Jul/2024:17:22:07.993601582 +0000] - INFO - dbmdb_public_dbmdb_import_main - import dsroot: Flushing caches...
[02/Jul/2024:17:22:07.995667599 +0000] - INFO - dbmdb_public_dbmdb_import_main - import dsroot: Closing files...
[02/Jul/2024:17:22:07.997941832 +0000] - INFO - dbmdb_public_dbmdb_import_main - import dsroot: Import complete.  Processed 1097 entries in 3 seconds. (365.67 entries/sec)
--
_______________________________________________
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue


--
--

389 Directory Server Development Team
--
_______________________________________________
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[389-users] Re: help with replication changelog size

Changed the subject of this email as this message was filtered into my
junk folder...

On 7/15/24 1:05 PM, kamakshya nayak wrote:
> I have a cascading multi setup 389-ds.
> I observed that on my supplier, hub and consumer are doing good in
> terms of replication.
> They are healthy.
> What I observe is supplier and hub is doing good with replication and
> size of changelog.db for one of the backend suffix is less.
> But, consumer changelog.db disk space utilization is more around 34 Gb
> and supplier and hub changelog.db disk space utilization  is 40 mb.
> What could be the issue and why it's utilizing that high disk space
> for changelog.db ?

What version of 389-ds-base are you using?

At first glance it would appear you have changelog trimming configured
on the supplier/hub but not on the consumer.  That could be the issue,
or it could be changelog compaction.  On the other hand a consumer does
not need a changelog so you could remove it's configuration.

Also I don't see a "changelog.db" file in my setup, so maybe you are on
a very old version?

Mark

>
> And, why a consumer is holding changelog.db file.. where could be the
> issue.
>
> Thank you.
>
--
Identity Management Development Team

--
_______________________________________________
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

Monday, July 15, 2024

[389-users] help

I have a cascading multi setup 389-ds.
I observed that on my supplier, hub and consumer are doing good in terms of replication.
They are healthy.
What I observe is supplier and hub is doing good with replication and size of changelog.db for one of the backend suffix is less.
But, consumer changelog.db disk space utilization is more around 34 Gb and supplier and hub changelog.db disk space utilization  is 40 mb.
What could be the issue and why it's utilizing that high disk space for changelog.db ?

And, why a consumer is holding changelog.db file.. where could be the issue.

Thank you.

[fedora-arm] [Fedocal] Reminder meeting : Fedora ARM & AArch64 status meeting

Dear all,

You are kindly invited to the meeting:
Fedora ARM & AArch64 status meeting on 2024-07-16 from 15:00:00 to 16:00:00 UTC

The meeting will be about:
Fedora ARM & AArch64 weekly status meeting.

Join us on Matrix at [https://matrix.to/#/#meeting-2:fedoraproject.org](https://matrix.to/#/#meeting-2:fedoraproject.org) through [chat.fedoraproject.org](through chat.fedoraproject.org).

More information available at:
[https://fedoraproject.org/wiki/Architectures/ARM](https://fedoraproject.org/wiki/Architectures/ARM)


Source: https://calendar.fedoraproject.org//meeting/10455/

--
_______________________________________________
arm mailing list -- arm@lists.fedoraproject.org
To unsubscribe send an email to arm-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/arm@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

Sunday, July 14, 2024

[Test-Announce]Fedora 41 Rawhide 20240714.n.0 nightly compose nominated for testing

Announcing the creation of a new nightly release validation test event
for Fedora 41 Rawhide 20240714.n.0. Please help run some tests for this
nightly compose if you have time. For more information on nightly
release validation testing, see:
https://fedoraproject.org/wiki/QA:Release_validation_test_plan

Notable package version changes:
pungi - 20240711.n.0: pungi-4.6.2-7.fc41.src, 20240714.n.0: pungi-4.6.3-1.fc41.src

Test coverage information for the current release can be seen at:
https://openqa.fedoraproject.org/testcase_stats/41

You can see all results, find testing instructions and image download
locations, and enter results on the Summary page:

https://fedoraproject.org/wiki/Test_Results:Fedora_41_Rawhide_20240714.n.0_Summary

The individual test result pages are:

https://fedoraproject.org/wiki/Test_Results:Fedora_41_Rawhide_20240714.n.0_Installation
https://fedoraproject.org/wiki/Test_Results:Fedora_41_Rawhide_20240714.n.0_Base
https://fedoraproject.org/wiki/Test_Results:Fedora_41_Rawhide_20240714.n.0_Server
https://fedoraproject.org/wiki/Test_Results:Fedora_41_Rawhide_20240714.n.0_Cloud
https://fedoraproject.org/wiki/Test_Results:Fedora_41_Rawhide_20240714.n.0_Desktop
https://fedoraproject.org/wiki/Test_Results:Fedora_41_Rawhide_20240714.n.0_Security_Lab

Thank you for testing!
--
Mail generated by relvalconsumer: https://pagure.io/fedora-qa/relvalconsumer
--
_______________________________________________
test-announce mailing list -- test-announce@lists.fedoraproject.org
To unsubscribe send an email to test-announce-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/test-announce@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue