From mvasko at cesnet.cz Wed Aug 15 08:20:16 2018 From: mvasko at cesnet.cz (=?utf-8?q?Michal_Va=C5=A1ko?=) Date: Wed, 15 Aug 2018 10:20:16 +0200 Subject: [sysrepo-devel] sysrepo libyang LYB format performance Message-ID: <699-5b73e280-1-14b96420@84790618> Hi everyone, I have finished (at least for now) libyang binary format development and integration into sysrepo. I have also done some benchmarks but I will not include callgrind outputs this time but real time taken. The reason is that it is ultimately what matters most and in this case the improvement in instruction count has proven to be significantly different from the improvement in real time. The values are an average from 5 measurements, the same data tree is used as always (50 000 list instances, ~750 000 individual nodes). sysrepo subscribe: JSON - 18,458s LYB - 1,764s sysrepo getconfig: JSON - 13,047s LYB - 10,043s sysrepo commit: JSON - 23,609s LYB - 3,838s The results for operations heavily dependent on parsing/printing data show a huge improvement. , which is mostly influenced by IPC speed has not improved that much and addressing this problem will be discussed in the next meeting. Specifically, I will present a new design of sysrepo, which will aim to be more reliable, simpler, and more efficient. I will also merge this LYB format support into libyang devel and fork all devel and master branches meaning they will no longer be compatible. While doing this, I will also change sysrepo default storing format to LYB. Can be easily changed during compilation by defining FILE_FORMAT_EXT with values xml or json. Regards, Michal From mvasko at cesnet.cz Fri Aug 24 14:10:20 2018 From: mvasko at cesnet.cz (=?utf-8?q?Michal_Va=C5=A1ko?=) Date: Fri, 24 Aug 2018 16:10:20 +0200 Subject: [sysrepo-devel] new generation of sysrepo Message-ID: <5b7f-5b801200-a7-408e4d0@232306574> Hello everyone, we have started working on the design of a completely new sysrepo. We try to present the biggest changes and overall functioning in the attached presentation and would like to ask everyone with a reasonable understanding of the current sysrepo and/or ideas for useful features or changes to go through it. We appreciate any feedback and will discuss this proposal internally on Wednesday. Thank you. Regards, Michal -------------- next part -------------- A non-text attachment was scrubbed... Name: new_sysrepo2.odp Type: application/vnd.oasis.opendocument.presentation Size: 770128 bytes Desc: not available URL: From bsderandrew at gmail.com Fri Aug 24 21:02:40 2018 From: bsderandrew at gmail.com (Andrew Collins) Date: Fri, 24 Aug 2018 15:02:40 -0600 Subject: [sysrepo-devel] new generation of sysrepo Message-ID: > we have started working on the design of a completely new sysrepo. I'm relatively new to sysrepo so can't comment on the changes, however as someone just starting a new project using sysrepo, any read on how significant the client side API changes will be? If I start a new project now on current sysrepo, will it be a full rewrite when sysrepo2 is available? From rkrejci at cesnet.cz Mon Aug 27 09:09:51 2018 From: rkrejci at cesnet.cz (=?UTF-8?B?UmFkZWsgS3JlasSNw60=?=) Date: Mon, 27 Aug 2018 11:09:51 +0200 Subject: [sysrepo-devel] new generation of sysrepo In-Reply-To: References: Message-ID: Hi, no, the intention is to preserve the current API as much as possible. We believe that it can be completely backward compatible, however we would like to do some changes to simplify/clarify the API (remove some unnecessary functions, rename function/enum to clarify its purpose, etc.). The biggest change in API, we think about, is probably support for the formats of the data passed between sysrepo and clients. Currently, there are 2 formats (sr_val_t and sr_node_t). Both are sysrepo specific despite internally we works with the data in libyang structures. It is too early to talk about details and before we decide, we want to do? a survey about using these two formats (sr_val_t is there from the beginning, sr_node_t was added later). However, what we are thinking about is a replacement/modification of the sr_node_t by something more natural to libyang. Regards, Radek Dne 24.8.2018 v 23:02 Andrew Collins napsal(a): >> we have started working on the design of a completely new sysrepo. > I'm relatively new to sysrepo so can't comment on the changes, however > as someone just starting a new project using sysrepo, any read on how > significant the client side API changes will be? If I start a new > project now on current sysrepo, will it be a full rewrite when > sysrepo2 is available? > _______________________________________________ > sysrepo-devel mailing list > sysrepo-devel at sysrepo.org > http://lists.sysrepo.org/listinfo/sysrepo-devel From mvasko at cesnet.cz Wed Aug 29 08:42:04 2018 From: mvasko at cesnet.cz (=?utf-8?q?Michal_Va=C5=A1ko?=) Date: Wed, 29 Aug 2018 10:42:04 +0200 Subject: [sysrepo-devel] sysrepo API change Message-ID: <364-5b865c80-11b-696a5400@13630730> Hi everyone, we would like to receive feedback on an API change we are planning. There is a good chance the majority of sysrepo clients will not be affected at all but that is exactly what we are trying to find out. Currently, there are 2 API variants for many functions. These functions are either without a suffix or end with "_subtree". Namely, sr_action_send(), sr_action_subscribe(), sr_event_notif_send(), sr_event_notif_subscribe(), sr_rpc_send(), sr_rpc_subscribe(). Additionally, there are pairs sr_get_item*() and sr_get_subtree*(). Functionality of these functions is the same except they return any nodes either in sr_val_t structures or sr_node_t structures. We would like to remove sr_node_t structures altogether and replace them with a libyang native data tree format struct lyd_node *. This format will also be much more efficient to work with than both sr_node_t and sr_val_t. TLDR; Do you have any clients using the subtree API variant of sysrepo functions (that work with sr_node_t)? If so, are you strongly against changing those structures to the libyang struct lyd_node *? All the information in sr_node_t can be also read from lyd_node but it would require some refactoring of your code. Thank you for any feedback. Regards, Michal From JoLandry at advaoptical.com Wed Aug 29 11:35:16 2018 From: JoLandry at advaoptical.com (Joan Landry) Date: Wed, 29 Aug 2018 11:35:16 +0000 Subject: [sysrepo-devel] sysrepo API change In-Reply-To: <364-5b865c80-11b-696a5400@13630730> References: <364-5b865c80-11b-696a5400@13630730> Message-ID: Will any refactoring be required for sr_val_t? If so can you provide an example of the type of refactoring that will be required. Thanks, Joan Landry -----Original Message----- From: sysrepo-devel On Behalf Of Michal Va?ko Sent: Wednesday, August 29, 2018 4:42 AM To: sysrepo-devel Subject: [sysrepo-devel] sysrepo API change Hi everyone, we would like to receive feedback on an API change we are planning. There is a good chance the majority of sysrepo clients will not be affected at all but that is exactly what we are trying to find out. Currently, there are 2 API variants for many functions. These functions are either without a suffix or end with "_subtree". Namely, sr_action_send(), sr_action_subscribe(), sr_event_notif_send(), sr_event_notif_subscribe(), sr_rpc_send(), sr_rpc_subscribe(). Additionally, there are pairs sr_get_item*() and sr_get_subtree*(). Functionality of these functions is the same except they return any nodes either in sr_val_t structures or sr_node_t structures. We would like to remove sr_node_t structures altogether and replace them with a libyang native data tree format struct lyd_node *. This format will also be much more efficient to work with than both sr_node_t and sr_val_t. TLDR; Do you have any clients using the subtree API variant of sysrepo functions (that work with sr_node_t)? If so, are you strongly against changing those structures to the libyang struct lyd_node *? All the information in sr_node_t can be also read from lyd_node but it would require some refactoring of your code. Thank you for any feedback. Regards, Michal _______________________________________________ sysrepo-devel mailing list sysrepo-devel at sysrepo.org http://lists.sysrepo.org/listinfo/sysrepo-devel From mvasko at cesnet.cz Wed Aug 29 11:40:08 2018 From: mvasko at cesnet.cz (=?utf-8?q?Michal_Va=C5=A1ko?=) Date: Wed, 29 Aug 2018 13:40:08 +0200 Subject: [sysrepo-devel] =?utf-8?b?Pz09P3V0Zi04P3E/ICBzeXNyZXBvIEFQSSBj?= =?utf-8?q?hange?= In-Reply-To: Message-ID: <21b8-5b868600-29-6ba5ac80@12184013> Hi Joan, no, sr_val_t will NOT be changed so no refactoring will be needed if you use only that. Regards, Michal On Wednesday, August 29, 2018 13:35 CEST, Joan Landry wrote: > Will any refactoring be required for sr_val_t? > If so can you provide an example of the type of refactoring that will be required. > Thanks, > Joan Landry > > -----Original Message----- > From: sysrepo-devel On Behalf Of Michal Va?ko > Sent: Wednesday, August 29, 2018 4:42 AM > To: sysrepo-devel > Subject: [sysrepo-devel] sysrepo API change > > Hi everyone, > we would like to receive feedback on an API change we are planning. There is a good chance the majority of sysrepo clients will not be affected at all but that is exactly what we are trying to find out. > > Currently, there are 2 API variants for many functions. These functions are either without a suffix or end with "_subtree". Namely, sr_action_send(), sr_action_subscribe(), sr_event_notif_send(), sr_event_notif_subscribe(), sr_rpc_send(), sr_rpc_subscribe(). Additionally, there are pairs sr_get_item*() and sr_get_subtree*(). Functionality of these functions is the same except they return any nodes either in sr_val_t structures or sr_node_t structures. We would like to remove sr_node_t structures altogether and replace them with a libyang native data tree format struct lyd_node *. This format will also be much more efficient to work with than both sr_node_t and sr_val_t. > > TLDR; Do you have any clients using the subtree API variant of sysrepo functions (that work with sr_node_t)? If so, are you strongly against changing those structures to the libyang struct lyd_node *? All the information in sr_node_t can be also read from lyd_node but it would require some refactoring of your code. > > Thank you for any feedback. > > Regards, > Michal > _______________________________________________ > sysrepo-devel mailing list > sysrepo-devel at sysrepo.org > http://lists.sysrepo.org/listinfo/sysrepo-devel