Sling Content Distribution: how to use "in-file" queue provider

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Sling Content Distribution: how to use "in-file" queue provider

Stefan Seifert
this presentation [1] mentions that it's possible to switch the Sling Content Distribution from using a Sling Jobs-based queue to an "in-file" or "in-memory" queue which is much faster.

but i've not found any hint in the documentation what steps are needed to switch to this queue. can you give some hints?

is it recommended to use this alternative queue in production (if the possible loss of some events up to the last checkpoint is acceptable)? is this the implementation of it [2]? the javadoc states "queue not persisted on the repository and therefore not usable for production."

stefan

[1] https://docs.adobe.com/ddc/en/gems/Troubleshooting-Sling-Content-Distribution.html
[2] https://github.com/apache/sling/blob/trunk/contrib/extensions/distribution/core/src/main/java/org/apache/sling/distribution/queue/impl/simple/SimpleDistributionQueue.java


Reply | Threaded
Open this post in threaded view
|

Re: Sling Content Distribution: how to use "in-file" queue provider

Timothee Maret-2
Hi,

Sorry for late reply, I was on PTO. The In-file queue is only available for
Forward agents.

To enable it, you need to set the 'queue.provider' property to 'In-file' in
your o.a.s.d.a.i.ForwardDistributionAgentFactory configuration.

The checkpoint mechanism should be implemented by [0] despite the comments
in SimpleDistributionQueue.

I don't think there's yet a customer with this setting. It should really be
used only if measurements prove that the queue is the bottleneck in your
setup, that throughput is a real issue in your deployment, that your
deployment can afford losing non checkpointed items and that you setup
custom operations to backup the queue.

Regards,

Timothee

[0]
https://github.com/apache/sling/blob/trunk/contrib/extensions/distribution/core/src/main/java/org/apache/sling/distribution/queue/impl/simple/SimpleDistributionQueueCheckpoint.java

2017-08-30 12:26 GMT+02:00 Stefan Seifert <[hidden email]>:

> this presentation [1] mentions that it's possible to switch the Sling
> Content Distribution from using a Sling Jobs-based queue to an "in-file" or
> "in-memory" queue which is much faster.
>
> but i've not found any hint in the documentation what steps are needed to
> switch to this queue. can you give some hints?
>
> is it recommended to use this alternative queue in production (if the
> possible loss of some events up to the last checkpoint is acceptable)? is
> this the implementation of it [2]? the javadoc states "queue not persisted
> on the repository and therefore not usable for production."
>
> stefan
>
> [1] https://docs.adobe.com/ddc/en/gems/Troubleshooting-Sling-
> Content-Distribution.html
> [2] https://github.com/apache/sling/blob/trunk/contrib/
> extensions/distribution/core/src/main/java/org/apache/
> sling/distribution/queue/impl/simple/SimpleDistributionQueue.java
>
>
>
Reply | Threaded
Open this post in threaded view
|

RE: Sling Content Distribution: how to use "in-file" queue provider

Stefan Seifert
thanks for the response! - we will keep this in mind.

stefan

>-----Original Message-----
>From: Timothee Maret [mailto:[hidden email]]
>Sent: Monday, September 11, 2017 11:34 AM
>To: [hidden email]
>Subject: Re: Sling Content Distribution: how to use "in-file" queue
>provider
>
>Hi,
>
>Sorry for late reply, I was on PTO. The In-file queue is only available for
>Forward agents.
>
>To enable it, you need to set the 'queue.provider' property to 'In-file' in
>your o.a.s.d.a.i.ForwardDistributionAgentFactory configuration.
>
>The checkpoint mechanism should be implemented by [0] despite the comments
>in SimpleDistributionQueue.
>
>I don't think there's yet a customer with this setting. It should really be
>used only if measurements prove that the queue is the bottleneck in your
>setup, that throughput is a real issue in your deployment, that your
>deployment can afford losing non checkpointed items and that you setup
>custom operations to backup the queue.
>
>Regards,
>
>Timothee
>
>[0]
>https://github.com/apache/sling/blob/trunk/contrib/extensions/distribution/
>core/src/main/java/org/apache/sling/distribution/queue/impl/simple/SimpleDi
>stributionQueueCheckpoint.java
>
>2017-08-30 12:26 GMT+02:00 Stefan Seifert <[hidden email]>:
>
>> this presentation [1] mentions that it's possible to switch the Sling
>> Content Distribution from using a Sling Jobs-based queue to an "in-file"
>or
>> "in-memory" queue which is much faster.
>>
>> but i've not found any hint in the documentation what steps are needed to
>> switch to this queue. can you give some hints?
>>
>> is it recommended to use this alternative queue in production (if the
>> possible loss of some events up to the last checkpoint is acceptable)? is
>> this the implementation of it [2]? the javadoc states "queue not
>persisted
>> on the repository and therefore not usable for production."
>>
>> stefan
>>
>> [1] https://docs.adobe.com/ddc/en/gems/Troubleshooting-Sling-
>> Content-Distribution.html
>> [2] https://github.com/apache/sling/blob/trunk/contrib/
>> extensions/distribution/core/src/main/java/org/apache/
>> sling/distribution/queue/impl/simple/SimpleDistributionQueue.java
>>
>>
>>