[sysrepo-devel] sysrepo performance with large datastore

Mislav Novakovic mislav.novakovic at sartura.hr
Wed Nov 15 10:33:00 UTC 2017


Hi Marco,

we are aware of the issues you are facing, the design philosophy by
now was to prioritize memory usage instead of performance because
Sysrepo/Netopeer2 was intended to be used on low memory devices.

Currently we are making some changes to the code base in order to make
performance improvements but I'm afraid the biggest bottleneck will
remain, which is the communication between Sysrepo and Netopeer2.

There are some options to fix this, like creating a new program which
combines Sysrepo and Netopeer2, and there is interest for fixing this
issue but at the moment we lack the resources for a project of this
scope.

There are some things you could do in order to improve performance:
1) combine NETCONF requests if possible, like add 10 list entries in
one NETCONF request instead of 10 separate ones.
2) make a tmpfs for the location /etc/sysrepo

Also can you make a Sysrepo feature ticket about this, there you can
explain your use case more in detail.

Regards,
Mislav

On Tue, Nov 14, 2017 at 9:16 AM, Augello Marco
<marco.augello at italtel.com> wrote:
> Hi all,
>
>
>
> we would use netopeer 2 for Italtel device configuration in cloud oriented
> applications
>
>
>
> We are testing sysrepod performance for our scopes.
>
>
>
> We noticed a significant performance degradation after the creation of
> thousands of data nodes.
>
> We always adding data nodes without deleting the old ones.
>
>
>
> It seems that the problem depends on the generation and the handling of the
> persistence file.
>
> When datastore file is large, also a single add operation is slow and more
> data nodes we adding more slow is the operation.
>
>
>
> For example, when the datastore is empty the first add is very fast, but
> after 4000 data nodes added, the next add takes 7 seconds and so on.
>
>
>
> Have you notice of this degradation? Maybe is our wrong configuration (In
> attach there is the yang model that we are testing at this moment)?
>
>
>
> Is in your roadmap an eventually change of the persistence handling?
> (different from single xml file)
>
>
>
> tell me if you need any other information.
>
>
>
> Thanks in advance,
>
> Sorry for my English J
>
>
>
> Marco
>
>
>
>
>
>
>
> Internet Email Confidentiality Footer
> ********************************************************************************************************************************************
> La presente comunicazione, con le informazioni in essa contenute e ogni
> documento o file allegato, e' rivolta unicamente alla/e persona/e cui e'
> indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete
> i destinatari/autorizzati siete avvisati che qualsiasi azione, copia,
> comunicazione, divulgazione o simili basate sul contenuto di tali
> informazioni e' vietata e potrebbe essere contro la legge (art. 616 C.P.,
> D.Lgs n. 196/2003 Codice in materia di protezione dei dati personali). Se
> avete ricevuto questa comunicazione per errore, vi preghiamo di darne
> immediata notizia al mittente e di distruggere il messaggio originale e ogni
> file allegato senza farne copia alcuna o riprodurne in alcun modo il
> contenuto. ***************** This e-mail and its attachments are intended
> for the addressee(s) only and are confidential and/or may contain legally
> privileged information. If you have received this message by mistake or are
> not one of the addressees above, you may take no action based on it, and you
> may not copy or show it to anyone; please reply to this e-mail and point out
> the error which has occurred.
> ********************************************************************************************************************************************
>
> _______________________________________________
> sysrepo-devel mailing list
> sysrepo-devel at sysrepo.org
> http://lists.sysrepo.org/listinfo/sysrepo-devel
>


More information about the sysrepo-devel mailing list