[389-devel] Re: Logging performance improvement

On Thu, 2016-06-30 at 20:53 -0600, Rich Megginson wrote:
> On 06/30/2016 08:14 PM, William Brown wrote:
> > On Thu, 2016-06-30 at 20:01 -0600, Rich Megginson wrote:
> >> On 06/30/2016 07:52 PM, William Brown wrote:
> >>> Hi,
> >>>
> >>> I've been thinking about this for a while, so I decided to dump my
> >>> thoughts to a document. I think I won't get to implementing this for a
> >>> while, but it would really help our server performance.
> >>>
> >>> http://www.port389.org/docs/389ds/design/logging-performance-improvement.html
> >> Looks good. Can we quantify the current log overhead?
> > Sure, I could probably sit down and work out a way to bench mark
> > this .....
> >
> > But without the alternative being written, hard to say. I could always
> > patch out logging and drop the lock in a hacked build so we can show
> > what "without logging contention" looks like?
>
> That's only one part of it - you'd have to figure out some way to get
> rid of the overhead of the formatting and flushing in the operation
> threads too.
>

This change would move the formatting into the thread that does the
write + flush.

Currently, we format the message in the operation thread, and send it to
the log buffer, and every now and again, an "unlucky" operation thread
has to stall to write the buffer ....

However, I'm proposing we send unformatted time struct + unformatted
message to the queue, then the actual thread which is separate does the
dequeue and format.

> I suppose you could just write it and see what happens.

Well, a hacked up test that has slapi_log_access return 0 immediately
(IE does not take the lock) showed a 6% performance improvement on my
laptop. On a higher cpu count system, this would exceed that number
easily, as more cpus == more cost to acquire the lock.

I've put the numbers on the design doc page, but I think with this, the
proof would be in the implementation. We also likely wouldn't see the
benefit of this as much until we remove other serialisation points in
the code IE ldbm vs lmdb, plugins that take locks, etc. So it's only a
6% possible gain now, but it may be 10% or more in the future in
combination with other efforts such as nunc-stans.

>
> >
> >>>
> >>> --
> >>> 389-devel mailing list
> >>> 389-devel@lists.fedoraproject.org
> >>> https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org
> >>
> >> --
> >> 389-devel mailing list
> >> 389-devel@lists.fedoraproject.org
> >> https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org
> >
> >
> > --
> > 389-devel mailing list
> > 389-devel@lists.fedoraproject.org
> > https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org
>
>
> --
> 389-devel mailing list
> 389-devel@lists.fedoraproject.org
> https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane