From jan.kundrat at cesnet.cz Tue Oct 2 15:07:33 2018 From: jan.kundrat at cesnet.cz (=?iso-8859-1?Q?Jan_Kundr=E1t?=) Date: Tue, 02 Oct 2018 17:07:33 +0200 Subject: [sysrepo-devel] new generation of sysrepo In-Reply-To: <33d1-5b90c700-3-5a619a00@135986086> References: <33d1-5b90c700-3-5a619a00@135986086> Message-ID: Hi, here's a bunch of comments from my part. We're using sysrepo (and the rest of the NETCONF stack) as an exclusive management interface to our optical devices. I will start with a personal wishlist related to general SW development, testing and cross-compiling. I wish sysrepo supported self-contained, portable repository locations. That would be super-useful for unit tests. Each test could then run in complete isolation from other parallel executions. There would be no extra setup and teardown (maybe just a `cp` and `rm`). In other words, please allow programmers to specify a "prefix" or "context" when initializing the first connection, and locate *all* data underneath that prefix. It is OK to have a system-wide default for "production use", but please treat another prefix as a first-class citizen. Lack of SR_EV_VERIFY sounds like a blocker for us. We are trying to push as much stuff as possible to XPath-based `must` conditions in our own models, but we've been hitting performance problems in the past, and the models are only getting more and more complex. Also, there are existing models written by people who put the validation logic into their NETCONF servers, i.e., there are rules not visible to clients via YANG. We must implement these models, and SR_EV_VERIFY is an important tool for that. I see in the updated presentation that it is probably not going to be removed, just renamed -- that's great. We "just" need to be able to reject a proposed change. I am a bit worried by the proposed use of SHM and zero-copy, but perhaps I'm just not understanding the slides. The current/legacy design uses GPB, a library for checking whether the data read from clients pass some rudimentory validation. How will different clients access SHM in the proposed next-gen design? Will they have direct access to, e.g., shared data structures? If the in-memory structures contain offsets or pointers, which part will check whether they are valid? What happens when a malicious client changes the data in an attempt to trigger a race condition (time of check vs time of use)? Right now, notifications leave much to be desired. Either they are one-off, or they are stored in an extremely inefficient format which basically requires an O(n^2) parsing overhead [1]. I wonder if sysrepo is actually a correct place for storing high-volume notifications. Perhaps looking at some existing message-bus system might make sense? Ops data subscriptions -- currently there are some limitations on how the code can be structured and how the YANG model can look like, especially when it comes to lists. Our implementation requests data from some low-level HW modules, and batching requests provides a notable performance boost. Therefore I prefer a single request for all ops data for a given module (or perhaps even modules?) with a list of requested XPath subtrees. That way our low-level code can use the most efficient method for obtaining the data. Changes to the startup datastore. Right now it requires a convoluted setup [2] where we're using `inotify`, listening for changes to the `foo.startup` data file. This means that we cannot reject such a modification, and that we have to use an external tool rather than hooking into sysrepo. Our use case is simple, e.g., a plugin which handles the `ietf-interfaces` model and pregenrates some persistent config to be applied during next reboot. Regarding locking, I don't have much to contribute here because our changes have so far only targeted a single DS. That said, it "feels wrong" if the commits are not actually atomic. If we can end up with a transaction which originally touches modules A and B, and it ends up only persisteng changes to, say, B, then that would be a bug in my opinion. Hope this helps, Jan [1] https://github.com/sysrepo/sysrepo/issues/735 [2] https://github.com/sysrepo/sysrepo/issues/954 From rkrejci at cesnet.cz Wed Oct 10 07:30:14 2018 From: rkrejci at cesnet.cz (=?UTF-8?B?UmFkZWsgS3JlasSNw60=?=) Date: Wed, 10 Oct 2018 09:30:14 +0200 Subject: [sysrepo-devel] new generation of sysrepo In-Reply-To: References: <33d1-5b90c700-3-5a619a00@135986086> Message-ID: <4dace672-623f-ac50-4a46-70b764bd7ce4@cesnet.cz> Hi Jan, as soon as michal will come back from his vacation, he'll surely add some more detailed answers, my opinions/understanding is inline... Dne 02. 10. 18 v 17:07 Jan Kundr?t napsal(a): > Hi, > here's a bunch of comments from my part. We're using sysrepo (and the rest of the NETCONF stack) as an exclusive management interface to our optical devices. > > I will start with a personal wishlist related to general SW development, testing and cross-compiling. I wish sysrepo supported self-contained, portable repository locations. That would be super-useful for unit tests. Each test could then run in complete isolation from other parallel executions. There would be no extra setup and teardown (maybe just a `cp` and `rm`). In other words, please allow programmers to specify a "prefix" or "context" when initializing the first connection, and locate *all* data underneath that prefix. It is OK to have a system-wide default for "production use", but please treat another prefix as a first-class citizen. I'm not against this idea > > Lack of SR_EV_VERIFY sounds like a blocker for us. We are trying to push as much stuff as possible to XPath-based `must` conditions in our own models, but we've been hitting performance problems in the past, and the models are only getting more and more complex. Also, there are existing models written by people who put the validation logic into their NETCONF servers, i.e., there are rules not visible to clients via YANG. We must implement these models, and SR_EV_VERIFY is an important tool for that. I see in the updated presentation that it is probably not going to be removed, just renamed -- that's great. We "just" need to be able to reject a proposed change. as you write at the end - it should be just rename of what we currently have. The point is that now we have VERIFY and then APPLY. In many cases, the verification must be actually done by applying changes to the device. So we want the first notification to be APPLY followed by DONE/ABORT. Anyway, during the first notification, you will be still able to reject the changes, so the behavior is planned to be kept as it is now. We can preserve even the names and just more clarify the descriptions, but we prefer even renaming the notifications. > > I am a bit worried by the proposed use of SHM and zero-copy, but perhaps I'm just not understanding the slides. The current/legacy design uses GPB, a library for checking whether the data read from clients pass some rudimentory validation. How will different clients access SHM in the proposed next-gen design? Will they have direct access to, e.g., shared data structures? If the in-memory structures contain offsets or pointers, which part will check whether they are valid? What happens when a malicious client changes the data in an attempt to trigger a race condition (time of check vs time of use)? with the proposed changes, we are actually moving to work more as /etc. The datastore (with granularity of the data for the different modules) access control will be done via standard Linux access rights (it is somehow already implemented in the current code). So only the processes with the allowed uid/gui will be allowed to access the specific data (the startup stored in files is clear, but the same mechanism should be used also for the shared memory segments). However, when the process has the access, it can do actually whatever it wants - which is the same case for the files in /etc. > > Right now, notifications leave much to be desired. Either they are one-off, or they are stored in an extremely inefficient format which basically requires an O(n^2) parsing overhead [1]. I wonder if sysrepo is actually a correct place for storing high-volume notifications. Perhaps looking at some existing message-bus system might make sense? Michal plans some changes, so it shouldn't be so demanding. Also with the design change of removing the main daemon, the processing demands will move to the specific processes which actually will work (store or load) with the notifications. However, looking how such a logging and working with a high-volume logs is done elsewhere, is a good point. > > Ops data subscriptions -- currently there are some limitations on how the code can be structured and how the YANG model can look like, especially when it comes to lists. Our implementation requests data from some low-level HW modules, and batching requests provides a notable performance boost. Therefore I prefer a single request for all ops data for a given module (or perhaps even modules?) with a list of requested XPath subtrees. That way our low-level code can use the most efficient method for obtaining the data. > I think that we were talking about some way to "cache" ops data, so you don't need to get each specific parameter from a device separately, but I'm not sure about the details right now. I think that there should be some announcement that we are now starting to get such data and then that we are done, so you can get all the data at the beginning and throw them out when it is done. > Changes to the startup datastore. Right now it requires a convoluted setup [2] where we're using `inotify`, listening for changes to the `foo.startup` data file. This means that we cannot reject such a modification, and that we have to use an external tool rather than hooking into sysrepo. Our use case is simple, e.g., a plugin which handles the `ietf-interfaces` model and pregenrates some persistent config to be applied during next reboot. I think that we were discussing that enhancing data subscriptions even to the startup repository is possible (despite I'm not a big fan of that). > > Regarding locking, I don't have much to contribute here because our changes have so far only targeted a single DS. That said, it "feels wrong" if the commits are not actually atomic. If we can end up with a transaction which originally touches modules A and B, and it ends up only persisteng changes to, say, B, then that would be a bug in my opinion. agree. The reason we want to have a separate locking for each YANG module data is to allow at least concurrent access to the different YANG module data. But when the changes affects more modules, all of them must be locked to start applying changes. Thank you for your comments, I really appreciate them. As I wrote, I expect that Michal will add more details or will correct me during this week. Regards, Radek > > Hope this helps, > Jan > > [1] https://github.com/sysrepo/sysrepo/issues/735 > [2] https://github.com/sysrepo/sysrepo/issues/954 > _______________________________________________ > sysrepo-devel mailing list > sysrepo-devel at sysrepo.org > http://lists.sysrepo.org/listinfo/sysrepo-devel From mvasko at cesnet.cz Thu Oct 11 07:24:30 2018 From: mvasko at cesnet.cz (=?utf-8?q?Michal_Va=C5=A1ko?=) Date: Thu, 11 Oct 2018 09:24:30 +0200 Subject: [sysrepo-devel] =?utf-8?b?Pz09P3V0Zi04P3E/ICBuZXcgZ2VuZXJhdGlv?= =?utf-8?q?n_of_sysrepo?= In-Reply-To: <4dace672-623f-ac50-4a46-70b764bd7ce4@cesnet.cz> Message-ID: <6445-5bbefa80-13-3ce4bd40@126230138> Hello, my input is inline. On Wednesday, October 10, 2018 09:30 CEST, Radek Krej?? wrote: > Hi Jan, > as soon as michal will come back from his vacation, he'll surely add some more detailed answers, my opinions/understanding is inline... > > Dne 02. 10. 18 v 17:07 Jan Kundr?t napsal(a): > > Hi, > > here's a bunch of comments from my part. We're using sysrepo (and the rest of the NETCONF stack) as an exclusive management interface to our optical devices. > > > > I will start with a personal wishlist related to general SW development, testing and cross-compiling. I wish sysrepo supported self-contained, portable repository locations. That would be super-useful for unit tests. Each test could then run in complete isolation from other parallel executions. There would be no extra setup and teardown (maybe just a `cp` and `rm`). In other words, please allow programmers to specify a "prefix" or "context" when initializing the first connection, and locate *all* data underneath that prefix. It is OK to have a system-wide default for "production use", but please treat another prefix as a first-class citizen. > > I'm not against this idea Sounds reasonable and should be fairly easily implemented in the code I already have. > > > > Lack of SR_EV_VERIFY sounds like a blocker for us. We are trying to push as much stuff as possible to XPath-based `must` conditions in our own models, but we've been hitting performance problems in the past, and the models are only getting more and more complex. Also, there are existing models written by people who put the validation logic into their NETCONF servers, i.e., there are rules not visible to clients via YANG. We must implement these models, and SR_EV_VERIFY is an important tool for that. I see in the updated presentation that it is probably not going to be removed, just renamed -- that's great. We "just" need to be able to reject a proposed change. > > as you write at the end - it should be just rename of what we currently have. The point is that now we have VERIFY and then APPLY. In many cases, the verification must be actually done by applying changes to the device. So we want the first notification to be APPLY followed by DONE/ABORT. Anyway, during the first notification, you will be still able to reject the changes, so the behavior is planned to be kept as it is now. We can preserve even the names and just more clarify the descriptions, but we prefer even renaming the notifications. What Radek wrote is accurate. I just want to add that however it will look at the end, every use-case supported now will definitely remain supported. > > > > > I am a bit worried by the proposed use of SHM and zero-copy, but perhaps I'm just not understanding the slides. The current/legacy design uses GPB, a library for checking whether the data read from clients pass some rudimentory validation. How will different clients access SHM in the proposed next-gen design? Will they have direct access to, e.g., shared data structures? If the in-memory structures contain offsets or pointers, which part will check whether they are valid? What happens when a malicious client changes the data in an attempt to trigger a race condition (time of check vs time of use)? > > with the proposed changes, we are actually moving to work more as /etc. The datastore (with granularity of the data for the different modules) access control will be done via standard Linux access rights (it is somehow already implemented in the current code). So only the processes with the allowed uid/gui will be allowed to access the specific data (the startup stored in files is clear, but the same mechanism should be used also for the shared memory segments). However, when the process has the access, it can do actually whatever it wants - which is the same case for the files in /etc. Exactly, the same problem affects any process having access to an important configuration file, it can corrupt it. It is up to the system administrator to set file access rights correctly and to install only trustworthy applications. > > > > > Right now, notifications leave much to be desired. Either they are one-off, or they are stored in an extremely inefficient format which basically requires an O(n^2) parsing overhead [1]. I wonder if sysrepo is actually a correct place for storing high-volume notifications. Perhaps looking at some existing message-bus system might make sense? > > Michal plans some changes, so it shouldn't be so demanding. Also with the design change of removing the main daemon, the processing demands will move to the specific processes which actually will work (store or load) with the notifications. However, looking how such a logging and working with a high-volume logs is done elsewhere, is a good point. The notification implementation in the current sysrepo is very simplistic so I do not think that it is too difficult to write an efficient one if keeping this property in mind from the start. By that I mean storing notifications in O(1) (naturally, adding a notification to an empty file will be much faster than to a 10 MB one but this can also be solved) and loading them in O(n). > > > > > Ops data subscriptions -- currently there are some limitations on how the code can be structured and how the YANG model can look like, especially when it comes to lists. Our implementation requests data from some low-level HW modules, and batching requests provides a notable performance boost. Therefore I prefer a single request for all ops data for a given module (or perhaps even modules?) with a list of requested XPath subtrees. That way our low-level code can use the most efficient method for obtaining the data. > > > > I think that we were talking about some way to "cache" ops data, so you don't need to get each specific parameter from a device separately, but I'm not sure about the details right now. I think that there should be some announcement that we are now starting to get such data and then that we are done, so you can get all the data at the beginning and throw them out when it is done. It is already possible to implement this with the (fairly) new request_id [3] parameter of the operational data callback. I mean to get all the data only once for one request. What you are proposing (passing request XPaths and getting all the data in one callback) changes the whole mechanism significantly, which we are trying to avoid. Nevertheless, we can discuss this more when the time is right (too soon right now). > > > Changes to the startup datastore. Right now it requires a convoluted setup [2] where we're using `inotify`, listening for changes to the `foo.startup` data file. This means that we cannot reject such a modification, and that we have to use an external tool rather than hooking into sysrepo. Our use case is simple, e.g., a plugin which handles the `ietf-interfaces` model and pregenrates some persistent config to be applied during next reboot. > > I think that we were discussing that enhancing data subscriptions even to the startup repository is possible (despite I'm not a big fan of that). Yeah, we are planning to support subscriptions not just for but also. > > > > > Regarding locking, I don't have much to contribute here because our changes have so far only targeted a single DS. That said, it "feels wrong" if the commits are not actually atomic. If we can end up with a transaction which originally touches modules A and B, and it ends up only persisteng changes to, say, B, then that would be a bug in my opinion. > > agree. The reason we want to have a separate locking for each YANG module data is to allow at least concurrent access to the different YANG module data. But when the changes affects more modules, all of them must be locked to start applying changes. Nothing to add. > > Thank you for your comments, I really appreciate them. As I wrote, I expect that Michal will add more details or will correct me during this week. > > Regards, > Radek > > > > > > Hope this helps, > > Jan > > > > [1] https://github.com/sysrepo/sysrepo/issues/735 > > [2] https://github.com/sysrepo/sysrepo/issues/954 > > _______________________________________________ > > sysrepo-devel mailing list > > sysrepo-devel at sysrepo.org > > http://lists.sysrepo.org/listinfo/sysrepo-devel Regards, Michal [3] https://github.com/sysrepo/sysrepo/blob/master/inc/sysrepo.h#L1847 From mvasko at cesnet.cz Wed Oct 17 08:01:45 2018 From: mvasko at cesnet.cz (=?utf-8?q?Michal_Va=C5=A1ko?=) Date: Wed, 17 Oct 2018 10:01:45 +0200 Subject: [sysrepo-devel] libssh critical security bug Message-ID: <2e49-5bc6ec80-35-24d89100@30334027> Hello, if you have not noticed new libssh versions 0.8.4 and 0.7.6 [1] and are using sysrepo with netopeer2-server in a publicly accessible network, we strongly encourage you to update your libssh as a critical security bugfix was included in the latest version. Regards, Michal [1] https://www.libssh.org/2018/10/16/libssh-0-8-4-and-0-7-6-security-and-bugfix-release/ From mvasko at cesnet.cz Wed Oct 17 12:25:16 2018 From: mvasko at cesnet.cz (=?utf-8?q?Michal_Va=C5=A1ko?=) Date: Wed, 17 Oct 2018 14:25:16 +0200 Subject: [sysrepo-devel] =?utf-8?q?Sysrepo_release_0=2E7=2E6?= Message-ID: <5e78-5bc72a00-41-469ce200@77866086> Hi everyone, we have released a new sysrepo version [1] but there are no major changes. The main reason for the release is that one libyang public flag was changed and it was no longer possible to compile the master branch of sysrepo with the master branch of libyang. Regards, Michal [1] https://github.com/sysrepo/sysrepo/releases/tag/v0.7.6 From vladimir.georgiev at telco.com Wed Oct 17 08:37:52 2018 From: vladimir.georgiev at telco.com (Vladimir Georgiev) Date: Wed, 17 Oct 2018 11:37:52 +0300 Subject: [sysrepo-devel] Issues with sysrepo's persist file Message-ID: Hi, I am writing to you, because we have problem using sysrepo. We are trying to use sysrepo as a database for machine configuration. We wrote several subscribers on Python that are services, started via systemd. During their start they subscribe itself to sysrepo. When all services/subscribers are subscribed and when there are some changes in sysrepo, all corresponding subscribers are called, as expected to be. The problem become visible on the next restart. The subscribers are unable to subscribe itself. Every request for subscribing raise exception "Sysrepo-internal error" After some investigations we saw that the persist file (in /etc/sysrepo/data) that is for corresponding root yang file, where is stored data of subscribers, socket, priority and etc. is broken (It is filled partially). It become in broken state during restart. The only fix, that we found for the situation, is removal of that file. But that is untill next changes in sysrepo and calling of subscribers, so after restart. I presume that the problem could be connected with simultaneous starting of services that lead to simultaneous subscribing requests/calls. Another thing is that the persist file is created with permissions 644 (all others persist files are 666), on the same time in sysrepo corresponding yang file is imported with permissions 666 (I do not know is there is a connection between both, but I do that remark, just in case) Sysrepo version: 0.7.4 libyang version: 0.15.166 Note: sysrepod is disabled as service. When it was enabled the persist file become more often broken, even after/during calling of subscribers. Do you have similar observation of similar problem? Any suggestions are welcome. Thank you! Best regards, Vladimir Georgiev -------------- next part -------------- An HTML attachment was scrubbed... URL: From mvasko at cesnet.cz Wed Oct 17 14:09:05 2018 From: mvasko at cesnet.cz (=?utf-8?q?Michal_Va=C5=A1ko?=) Date: Wed, 17 Oct 2018 16:09:05 +0200 Subject: [sysrepo-devel] =?utf-8?b?Pz09P3V0Zi04P3E/ICBJc3N1ZXMgd2l0aCBz?= =?utf-8?q?ysrepo=27s_persist_file?= In-Reply-To: Message-ID: <3d66-5bc74280-4d-740eec80@171404663> Hi Vladimir, On Wednesday, October 17, 2018 10:37 CEST, Vladimir Georgiev wrote: > Hi, > I am writing to you, because we have problem using sysrepo. > We are trying to use sysrepo as a database for machine configuration. > We wrote several subscribers on Python that are services, started via > systemd. During their start they subscribe itself to sysrepo. > When all services/subscribers are subscribed and when there are some > changes in sysrepo, all corresponding subscribers are called, as expected > to be. > > The problem become visible on the next restart. The subscribers are unable > to subscribe itself. Every request for subscribing raise exception > "Sysrepo-internal error" > After some investigations we saw that the persist file (in > /etc/sysrepo/data) that is for corresponding root yang file, where is > stored data of subscribers, socket, priority and etc. is broken (It is > filled partially). > It become in broken state during restart. The only fix, that we found for > the situation, is removal of that file. But that is untill next changes in > sysrepo and calling of subscribers, so after restart. We have not experienced anything like you described. My first advice is to update sysrepo and libyang to most up-to-date versions, it would be difficult to help you further without that. Now, what do you mean by restart, you simply terminate sysrepod and execute it again? You checked the persist file right after sysrepod restarted and it was corrupted? Was it already corrupted after sysrepod was terminated? > I presume that the problem could be connected with simultaneous starting of > services that lead to simultaneous subscribing requests/calls. Perhaps, but each of these persist files should be file-system-locked for this not to be an issue. > Another thing is that the persist file is created with permissions 644 (all > others persist files are 666), on the same time in sysrepo corresponding > yang file is imported with permissions 666 (I do not know is there is a > connection between both, but I do that remark, just in case) That is definitely suspicious and you should try to find out how is this file created if you can. sysrepoctl should set 0666 permissions on all the files it creates. I am not able to help you more, maybe someone else from sysrepo members can? Maybe the python bindings are to blame? Regards, Michal From mislav.novakovic at sartura.hr Wed Oct 17 14:48:36 2018 From: mislav.novakovic at sartura.hr (Mislav Novakovic) Date: Wed, 17 Oct 2018 16:48:36 +0200 Subject: [sysrepo-devel] ?==?utf-8?q? Issues with sysrepo's persist file In-Reply-To: <3d66-5bc74280-4d-740eec80@171404663> References: <3d66-5bc74280-4d-740eec80@171404663> Message-ID: Hi Vladimir, On Wed, Oct 17, 2018 at 4:09 PM Michal Va?ko wrote: > > Hi Vladimir, > > On Wednesday, October 17, 2018 10:37 CEST, Vladimir Georgiev wrote: > > > Hi, > > I am writing to you, because we have problem using sysrepo. > > We are trying to use sysrepo as a database for machine configuration. > > We wrote several subscribers on Python that are services, started via > > systemd. During their start they subscribe itself to sysrepo. > > When all services/subscribers are subscribed and when there are some > > changes in sysrepo, all corresponding subscribers are called, as expected > > to be. > > > > The problem become visible on the next restart. The subscribers are unable > > to subscribe itself. Every request for subscribing raise exception > > "Sysrepo-internal error" > > After some investigations we saw that the persist file (in > > /etc/sysrepo/data) that is for corresponding root yang file, where is > > stored data of subscribers, socket, priority and etc. is broken (It is > > filled partially). > > It become in broken state during restart. The only fix, that we found for > > the situation, is removal of that file. But that is untill next changes in > > sysrepo and calling of subscribers, so after restart. > > We have not experienced anything like you described. My first advice is to update sysrepo and libyang to most up-to-date versions, it would be difficult to help you further without that. Now, what do you mean by restart, you simply terminate sysrepod and execute it again? You checked the persist file right after sysrepod restarted and it was corrupted? Was it already corrupted after sysrepod was terminated? > > > I presume that the problem could be connected with simultaneous starting of > > services that lead to simultaneous subscribing requests/calls. > > Perhaps, but each of these persist files should be file-system-locked for this not to be an issue. > > > Another thing is that the persist file is created with permissions 644 (all > > others persist files are 666), on the same time in sysrepo corresponding > > yang file is imported with permissions 666 (I do not know is there is a > > connection between both, but I do that remark, just in case) > > That is definitely suspicious and you should try to find out how is this file created if you can. sysrepoctl should set 0666 permissions on all the files it creates. > > I am not able to help you more, maybe someone else from sysrepo members can? Maybe the python bindings are to blame? Please update sysrepo, we made some recent changes to the C++ and Python bindings. If the issue persists with the new sysrepo version please can you send us your systemd files. There are systemd examples for sysrepo at: https://github.com/sysrepo/sysrepo/tree/master/deploy/systemd . A long time ago we did have a similar issue once on an embedded device, where sysrepod would crash at startup due to low RAM and some XML files would get corrupted. In your case I suspect that at boot time sysrepod crashes and restarts for some reason. Regards, Mislav _______________________________________________ > sysrepo-devel mailing list > sysrepo-devel at sysrepo.org > http://lists.sysrepo.org/listinfo/sysrepo-devel From vladimir.georgiev at telco.com Wed Oct 17 14:48:28 2018 From: vladimir.georgiev at telco.com (Vladimir Georgiev) Date: Wed, 17 Oct 2018 17:48:28 +0300 Subject: [sysrepo-devel] Issues with sysrepo's persist file In-Reply-To: <3d66-5bc74280-4d-740eec80@171404663> References: <3d66-5bc74280-4d-740eec80@171404663> Message-ID: Hi Michal, Thank you for your fast reply. Under restart I mean reboot of the whole system "sudo reboot". Currently on our system "sysrepod" is disabled with "systemctl disable sysrepod" and is not started as daemon during boot. We did that because we experienced more often corruption of the persist file, when sysrepod is active. In taht case the persist file become corrupted on random basis and we did not catch reproducibility with exact steps. That was the reason first we to presume that it could be some conflict between library calls and sysrepod and to disable sysrepod. But corruption of the persist file continue to be present even after the deactivation of sysrepod. The difference is that now it is with reproducibility steps. As I explained our subscribers are in fact services/daemons. On the first boot all of them subscribe them to sysrepo successful. Then when we change some things in sysrepo that will lead to activation of at least two subscribers (only one does is not a problem), on the next reboot - new cycle for subscription, the persist file become corrupted and all rest subscription request are rejected with exception "sysrepo-internal error". I want to remark here, that before that reboot, the system is operative long time without problem. Today I've made serialization of execution of daemons that are subscribers, just in case. Now the situation is more stable - at least the persist file does not become corrupted, but we continue to get exceptions "sysrepo-internal error" from time to time. Bu now simple restarting of failed daemons(subscribers) that failed because of these exceptions, do the work. About permissions indeed I am also confused, because according the code persist files should be with permissions 0666. As I saw, there is only one place in the code where the permissions are 0644 during file creation, but I am not sure how much that is related to that case. Yes, it could be connected with Python bindings. We continue to search the problem. So any ideas are welcome. About the new version, we finally will be forced to try it, but because of that will consume some time, that for the moment we try to avoid, we tried to stabilise the current situation. Anyway - Thank you for your help! Best regards, Vladimir Georgiev On Wed, Oct 17, 2018 at 5:09 PM Michal Va?ko wrote: > Hi Vladimir, > > On Wednesday, October 17, 2018 10:37 CEST, Vladimir Georgiev < > vladimir.georgiev at telco.com> wrote: > > > Hi, > > I am writing to you, because we have problem using sysrepo. > > We are trying to use sysrepo as a database for machine configuration. > > We wrote several subscribers on Python that are services, started via > > systemd. During their start they subscribe itself to sysrepo. > > When all services/subscribers are subscribed and when there are some > > > changes in sysrepo, all corresponding subscribers are called, as expected > > to be. > > > > The problem become visible on the next restart. The subscribers are > unable > > to subscribe itself. Every request for subscribing raise exception > > "Sysrepo-internal error" > > After some investigations we saw that the persist file (in > > /etc/sysrepo/data) that is for corresponding root yang file, where is > > stored data of subscribers, socket, priority and etc. is broken (It is > > filled partially). > > It become in broken state during restart. The only fix, that we found for > > the situation, is removal of that file. But that is untill next changes > in > > sysrepo and calling of subscribers, so after restart. > > We have not experienced anything like you described. My first advice is to > update sysrepo and libyang to most up-to-date versions, it would be > difficult to help you further without that. Now, what do you mean by > restart, you simply terminate sysrepod and execute it again? You checked > the persist file right after sysrepod restarted and it was corrupted? Was > it already corrupted after sysrepod was terminated? > > > I presume that the problem could be connected with simultaneous starting > of > > services that lead to simultaneous subscribing requests/calls. > > Perhaps, but each of these persist files should be file-system-locked for > this not to be an issue. > > > Another thing is that the persist file is created with permissions 644 > (all > > others persist files are 666), on the same time in sysrepo corresponding > > yang file is imported with permissions 666 (I do not know is there is a > > connection between both, but I do that remark, just in case) > > That is definitely suspicious and you should try to find out how is this > file created if you can. sysrepoctl should set 0666 permissions on all the > files it creates. > > I am not able to help you more, maybe someone else from sysrepo members > can? Maybe the python bindings are to blame? > > Regards, > Michal > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jan.kundrat at cesnet.cz Fri Oct 19 12:45:39 2018 From: jan.kundrat at cesnet.cz (=?iso-8859-1?Q?Jan_Kundr=E1t?=) Date: Fri, 19 Oct 2018 14:45:39 +0200 Subject: [sysrepo-devel] Issues with sysrepo's persist file In-Reply-To: References: <3d66-5bc74280-4d-740eec80@171404663> Message-ID: On st?eda 17. ??jna 2018 16:48:28 CEST, Vladimir Georgiev wrote: > Currently on our system "sysrepod" is disabled with "systemctl disable > sysrepod" and is not started as daemon during boot. We did that because we > experienced more often corruption of the persist file, when sysrepod is > active. In taht case the persist file become corrupted on random basis and > we did not catch reproducibility with exact steps. That was the reason > first we to presume that it could be some conflict between library calls > and sysrepod and to disable sysrepod. > > But corruption of the persist file continue to be present even after the > deactivation of sysrepod. The difference is that now it is with > reproducibility steps. That sounds as if you now switched from a single systrepod instance managed by systemd to an approach where the sysrepo client library auto-starts a `sysrepod` process once it is first needed. That is not a correct path to take. In fact, I believe that this auto-starting should be disabled on the API level, or at least made explicit and to default to not fork the daemon in the background (see below for technical reasoning). What I would try if I were you is the following: - update to recent releases of the whole stack, including libyang, netopeer2-server, sysrepo, libnetconf2 - ensure that every user of sysrepod is managed by systemd, that it has a Requires=sysrepod.service and After=sysrepod.service stanzas in their respective .service/.unit files - audit your code that you open all of your sysrepo client sessions with a flag SR_CONN_DAEMON_REQUIRED That way you get determinism in the startup sequence, and you can be sure that no matter how badly the system behaves, the `sysrepod` process still runs with correct UID, ACLs, selinux labels (if applicable), Linux kernel namespace setup, sane environment, etc etc etc. With kind regards, Jan From bsderandrew at gmail.com Fri Oct 19 20:09:30 2018 From: bsderandrew at gmail.com (Andrew Collins) Date: Fri, 19 Oct 2018 14:09:30 -0600 Subject: [sysrepo-devel] A few simple sysrepo questions Message-ID: I'm looking into sysrepo for a new project and I have a few, hopefully simple, questions. I've read the docs and the answer wasn't immediately obvious to me, but if RTFM is the proper answer here, let me know. Anyway, I'm looking into sysrepo as both a configuration store as well as a form of yang-based-IPC. That is, not only will it store configuration coming from an external source, but when two services wish to communicate, one will do a "sr_set_item" to a data store in volatile storage (RAM), and another would register a change handler to respond to and handle the event. I realize there exists the yang "rpc" definition, and associated sysrepo subsystem, but I'm interested in a more persistent alternative, one which doesn't get stored to filesystem, but is not as ephemeral as a one time message. The goal here is to avoid the case where a service is unavailable/crashed/being restarted, and an important RPC is sent but not handled. I'm relatively new to all these systems so I may be taking the wrong approach here, but I'm interested if anyone has any ideas here. Another unrelated question I have is around the efficiency of state data handling. In a complex system there may be a variety of state elements which can be pushed, rather than pulled (mainly items that rarely change). That is, for the sake of efficiency, rather than registering a data provider subscription, simply doing an "sr_set_item" whenever the state element needs to be changed. Can state elements be just "set", or is a data provider always required? Thanks, Andrew From mvasko at cesnet.cz Mon Oct 22 06:59:40 2018 From: mvasko at cesnet.cz (=?utf-8?q?Michal_Va=C5=A1ko?=) Date: Mon, 22 Oct 2018 08:59:40 +0200 Subject: [sysrepo-devel] =?utf-8?b?Pz09P3V0Zi04P3E/ICBBIGZldyBzaW1wbGUg?= =?utf-8?q?sysrepo_questions?= In-Reply-To: Message-ID: <699f-5bcd7580-11-17e9de40@104858869> Hi, On Friday, October 19, 2018 22:09 CEST, Andrew Collins wrote: > I'm looking into sysrepo for a new project and I have a few, hopefully > simple, questions. I've read the docs and the answer wasn't > immediately obvious to me, but if RTFM is the proper answer here, let > me know. > > Anyway, I'm looking into sysrepo as both a configuration store as well > as a form of yang-based-IPC. That is, not only will it store > configuration coming from an external source, but when two services > wish to communicate, one will do a "sr_set_item" to a data store in > volatile storage (RAM), and another would register a change handler to > respond to and handle the event. > > I realize there exists the yang "rpc" definition, and associated > sysrepo subsystem, but I'm interested in a more persistent > alternative, one which doesn't get stored to filesystem, but is not as > ephemeral as a one time message. The goal here is to avoid the case > where a service is unavailable/crashed/being restarted, and an > important RPC is sent but not handled. I'm relatively new to all > these systems so I may be taking the wrong approach here, but I'm > interested if anyone has any ideas here. I think this is a use-case covered by neither YANG nor sysrepo so I would advise against doing this. If you still want to use sysrepo like this, there is no better way than what you described, I think. > Another unrelated question I have is around the efficiency of state > data handling. In a complex system there may be a variety of state > elements which can be pushed, rather than pulled (mainly items that > rarely change). That is, for the sake of efficiency, rather than > registering a data provider subscription, simply doing an > "sr_set_item" whenever the state element needs to be changed. Can > state elements be just "set", or is a data provider always required? No, you must act as a data provider. However, this provider can simply cache the state data and the actual provider can just "push" changes to this cache. Also, this use-case should be directly supported in the new sysrepo generation with NMDA [1]. Regards, Michal [1] https://tools.ietf.org/html/rfc8342 From bsderandrew at gmail.com Mon Oct 22 21:50:13 2018 From: bsderandrew at gmail.com (Andrew Collins) Date: Mon, 22 Oct 2018 15:50:13 -0600 Subject: [sysrepo-devel] A few simple sysrepo questions In-Reply-To: <699f-5bcd7580-11-17e9de40@104858869> References: <699f-5bcd7580-11-17e9de40@104858869> Message-ID: > I think this is a use-case covered by neither YANG nor sysrepo so I would advise against doing this. If you still want to use sysrepo like this, there is no better way than what you described, I think. +cc list this time... Thank your for the helpful responses Michal, in reading the NMDA RFC I ran across https://tools.ietf.org/html/rfc8342#section-5.2 which seems to be similar to what I am looking for, that is an ephemeral data store: The model recognizes the need for dynamic configuration datastores that are, by definition, not part of the persistent configuration of a device. In some contexts, these have been termed "ephemeral datastores", since the information is ephemeral, i.e., lost upon reboot. o dynamic configuration datastore: A configuration datastore holding configuration obtained dynamically during the operation of a device through interaction with other systems, rather than through one of the conventional configuration datastores. Is this part of the upcoming NMDA support in next-gen sysrepo? Thanks, Andrew On Mon, Oct 22, 2018 at 12:59 AM Michal Va?ko wrote: > > Hi, > > On Friday, October 19, 2018 22:09 CEST, Andrew Collins wrote: > > > I'm looking into sysrepo for a new project and I have a few, hopefully > > simple, questions. I've read the docs and the answer wasn't > > immediately obvious to me, but if RTFM is the proper answer here, let > > me know. > > > > Anyway, I'm looking into sysrepo as both a configuration store as well > > as a form of yang-based-IPC. That is, not only will it store > > configuration coming from an external source, but when two services > > wish to communicate, one will do a "sr_set_item" to a data store in > > volatile storage (RAM), and another would register a change handler to > > respond to and handle the event. > > > > I realize there exists the yang "rpc" definition, and associated > > sysrepo subsystem, but I'm interested in a more persistent > > alternative, one which doesn't get stored to filesystem, but is not as > > ephemeral as a one time message. The goal here is to avoid the case > > > where a service is unavailable/crashed/being restarted, and an > > important RPC is sent but not handled. I'm relatively new to all > > these systems so I may be taking the wrong approach here, but I'm > > interested if anyone has any ideas here. > > I think this is a use-case covered by neither YANG nor sysrepo so I would advise against doing this. If you still want to use sysrepo like this, there is no better way than what you described, I think. > > > Another unrelated question I have is around the efficiency of state > > data handling. In a complex system there may be a variety of state > > elements which can be pushed, rather than pulled (mainly items that > > rarely change). That is, for the sake of efficiency, rather than > > registering a data provider subscription, simply doing an > > "sr_set_item" whenever the state element needs to be changed. Can > > state elements be just "set", or is a data provider always required? > > > No, you must act as a data provider. However, this provider can simply cache the state data and the actual provider can just "push" changes to this cache. Also, this use-case should be directly supported in the new sysrepo generation with NMDA [1]. > > Regards, > Michal > > [1] https://tools.ietf.org/html/rfc8342 From mvasko at cesnet.cz Tue Oct 23 06:34:08 2018 From: mvasko at cesnet.cz (=?utf-8?q?Michal_Va=C5=A1ko?=) Date: Tue, 23 Oct 2018 08:34:08 +0200 Subject: [sysrepo-devel] =?utf-8?b?Pz09P3V0Zi04P3E/ICBBIGZldyBzaW1wbGUg?= =?utf-8?q?sysrepo_questions?= In-Reply-To: Message-ID: Hi Andrew, actually, we did not plan to support this datastore but it seems it is exactly your use-case. We did not pay much attention to it as it does not interact with other datastores. So, once the new generation sysrepo is finished with all currently-planned features (at least half a year from now) we probably can implement this. Regards, Michal On Monday, October 22, 2018 23:50 CEST, Andrew Collins wrote: > > I think this is a use-case covered by neither YANG nor sysrepo so I would advise against doing this. If you still want to use sysrepo like this, there is no better way than what you described, I think. > > +cc list this time... > > Thank your for the helpful responses Michal, in reading the NMDA RFC I > ran across https://tools.ietf.org/html/rfc8342#section-5.2 which seems > to be similar to what I am looking for, that is an ephemeral data > store: > > The model recognizes the need for dynamic configuration datastores > that are, by definition, not part of the persistent configuration of > a device. In some contexts, these have been termed "ephemeral > datastores", since the information is ephemeral, i.e., lost upon > reboot. > > o dynamic configuration datastore: A configuration datastore holding > configuration obtained dynamically during the operation of a > device through interaction with other systems, rather than through > one of the conventional configuration datastores. > > Is this part of the upcoming NMDA support in next-gen sysrepo? > > Thanks, > Andrew > On Mon, Oct 22, 2018 at 12:59 AM Michal Va?ko wrote: > > > > Hi, > > > > On Friday, October 19, 2018 22:09 CEST, Andrew Collins wrote: > > > > > I'm looking into sysrepo for a new project and I have a few, hopefully > > > simple, questions. I've read the docs and the answer wasn't > > > immediately obvious to me, but if RTFM is the proper answer here, let > > > me know. > > > > > > Anyway, I'm looking into sysrepo as both a configuration store as well > > > as a form of yang-based-IPC. That is, not only will it store > > > configuration coming from an external source, but when two services > > > wish to communicate, one will do a "sr_set_item" to a data store in > > > volatile storage (RAM), and another would register a change handler to > > > respond to and handle the event. > > > > > > I realize there exists the yang "rpc" definition, and associated > > > sysrepo subsystem, but I'm interested in a more persistent > > > alternative, one which doesn't get stored to filesystem, but is not as > > > ephemeral as a one time message. The goal here is to avoid the case > > > > > where a service is unavailable/crashed/being restarted, and an > > > important RPC is sent but not handled. I'm relatively new to all > > > these systems so I may be taking the wrong approach here, but I'm > > > interested if anyone has any ideas here. > > > > I think this is a use-case covered by neither YANG nor sysrepo so I would advise against doing this. If you still want to use sysrepo like this, there is no better way than what you described, I think. > > > > > Another unrelated question I have is around the efficiency of state > > > data handling. In a complex system there may be a variety of state > > > elements which can be pushed, rather than pulled (mainly items that > > > rarely change). That is, for the sake of efficiency, rather than > > > registering a data provider subscription, simply doing an > > > "sr_set_item" whenever the state element needs to be changed. Can > > > state elements be just "set", or is a data provider always required? > > > > > > No, you must act as a data provider. However, this provider can simply cache the state data and the actual provider can just "push" changes to this cache. Also, this use-case should be directly supported in the new sysrepo generation with NMDA [1]. > > > > Regards, > > Michal > > > > [1] https://tools.ietf.org/html/rfc8342 From vladimir.georgiev at telco.com Tue Oct 23 11:03:13 2018 From: vladimir.georgiev at telco.com (Vladimir Georgiev) Date: Tue, 23 Oct 2018 14:03:13 +0300 Subject: [sysrepo-devel] Issues with sysrepo's persist file In-Reply-To: <3d66-5bc74280-4d-740eec80@171404663> References: <3d66-5bc74280-4d-740eec80@171404663> Message-ID: Hi again, I want to thank you for your reply. Unfortunately we continue our saga with subscribers. We were able to isolate strange situation. There are no subscribers to sysrepo. Sysrepod started with debug options (-d -l 4). During execution of "sysrepocfg --get /ucpe:config//* -f json", we got error that there are no subscribers (that is as expected), but on sysrepod logging we are seeing strange WRN message: [DBG] New message of size 29 bytes received. [DBG] Processing session_stop request (session id=96703980). [DBG] RP session stop, session id=96703980. *[WRN] There are some (1) unprocessed messages for the session id=96703980 when session stop has been requested, this can lead to unspecified behavior - check RP caller code!!!* [DBG] Sending 33 bytes of data. [DBG] 33 bytes of data sent. [INF] Dropping session id=96703980. [DBG] fd 6 readable (revents 1) [DBG] Peer on fd 6 disconnected. [INF] Closing the connection 0x5555557d6df0. We got a lot of these *WRN* messages during the subscribing of our subscribers to sysrepo (when sysrepod is active), but recently we were able to isolate such message, without our subscribers. Only sysrepod and sysrepocfg Is that behave is normal? Also I want to ask few other things: - At the moment we create a connection, then create session then do subscribe the subscriber. But both connection and session are open till stopping of the subscriber from systemd. Is that is the right way, or session should be closed and opened only when it is needed? - Do you have any idea why when we try to make connection and then session from another subscriber (one of both functions throw exception with sysrepo-internal error) . Like resource is locked. Now we have retry mechanism, but it is like brute force - try until succeed, but that is not elegant How to recognize when resource is free, so only then to do try to subscribe? Thanks. Best regards, Vladimir Georgiev On Wed, Oct 17, 2018 at 5:09 PM Michal Va?ko wrote: > Hi Vladimir, > > On Wednesday, October 17, 2018 10:37 CEST, Vladimir Georgiev < > vladimir.georgiev at telco.com> wrote: > > > Hi, > > I am writing to you, because we have problem using sysrepo. > > We are trying to use sysrepo as a database for machine configuration. > > We wrote several subscribers on Python that are services, started via > > systemd. During their start they subscribe itself to sysrepo. > > When all services/subscribers are subscribed and when there are some > > > changes in sysrepo, all corresponding subscribers are called, as expected > > to be. > > > > The problem become visible on the next restart. The subscribers are > unable > > to subscribe itself. Every request for subscribing raise exception > > "Sysrepo-internal error" > > After some investigations we saw that the persist file (in > > /etc/sysrepo/data) that is for corresponding root yang file, where is > > stored data of subscribers, socket, priority and etc. is broken (It is > > filled partially). > > It become in broken state during restart. The only fix, that we found for > > the situation, is removal of that file. But that is untill next changes > in > > sysrepo and calling of subscribers, so after restart. > > We have not experienced anything like you described. My first advice is to > update sysrepo and libyang to most up-to-date versions, it would be > difficult to help you further without that. Now, what do you mean by > restart, you simply terminate sysrepod and execute it again? You checked > the persist file right after sysrepod restarted and it was corrupted? Was > it already corrupted after sysrepod was terminated? > > > I presume that the problem could be connected with simultaneous starting > of > > services that lead to simultaneous subscribing requests/calls. > > Perhaps, but each of these persist files should be file-system-locked for > this not to be an issue. > > > Another thing is that the persist file is created with permissions 644 > (all > > others persist files are 666), on the same time in sysrepo corresponding > > yang file is imported with permissions 666 (I do not know is there is a > > connection between both, but I do that remark, just in case) > > That is definitely suspicious and you should try to find out how is this > file created if you can. sysrepoctl should set 0666 permissions on all the > files it creates. > > I am not able to help you more, maybe someone else from sysrepo members > can? Maybe the python bindings are to blame? > > Regards, > Michal > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mvasko at cesnet.cz Tue Oct 23 11:33:51 2018 From: mvasko at cesnet.cz (=?utf-8?q?Michal_Va=C5=A1ko?=) Date: Tue, 23 Oct 2018 13:33:51 +0200 Subject: [sysrepo-devel] =?utf-8?b?Pz09P3V0Zi04P3E/ICBJc3N1ZXMgd2l0aCBz?= =?utf-8?q?ysrepo=27s_persist_file?= In-Reply-To: Message-ID: <5ad2-5bcf0700-b-505af780@248564044> Hi, sorry to hear that but I do not think we will be of much help. On Tuesday, October 23, 2018 13:03 CEST, Vladimir Georgiev wrote: > Hi again, > > I want to thank you for your reply. Unfortunately we continue our saga with > subscribers. > > We were able to isolate strange situation. There are no subscribers to > sysrepo. Sysrepod started with debug options (-d -l 4). > During execution of "sysrepocfg --get /ucpe:config//* -f json", we got > error that there are no subscribers (that is as expected), but on sysrepod > logging we are seeing strange WRN message: > > [DBG] New message of size 29 bytes received. > [DBG] Processing session_stop request (session id=96703980). > [DBG] RP session stop, session id=96703980. > *[WRN] There are some (1) unprocessed messages for the session id=96703980 > when session stop has been requested, this can lead to unspecified behavior > - check RP caller code!!!* > [DBG] Sending 33 bytes of data. > [DBG] 33 bytes of data sent. > [INF] Dropping session id=96703980. > [DBG] fd 6 readable (revents 1) > [DBG] Peer on fd 6 disconnected. > [INF] Closing the connection 0x5555557d6df0. > > > We got a lot of these *WRN* messages during the subscribing of our > subscribers to sysrepo (when sysrepod is active), but recently we were able > to isolate such message, without our subscribers. Only sysrepod and > sysrepocfg > > Is that behave is normal? No, whenever you see that message, something is seriously wrong. If this always happens to you right after starting sysrepod, there is some persistent internal problem so I would suggest to remove the whole repository and reinstall sysrepo. > Also I want to ask few other things: > - At the moment we create a connection, then create session then do > subscribe the subscriber. But both connection and session are open till > stopping of the subscriber from systemd. Is that is the right way, or > session should be closed and opened only when it is needed? It is fine this way. > - Do you have any idea why when we try to make connection and then session > from another subscriber (one of both functions throw exception with > sysrepo-internal error) . Like resource is locked. Now we have retry > mechanism, but it is like brute force - try until succeed, but that is not > elegant How to recognize when resource is free, so only then to do try to > subscribe? Unfortunately, there is no better way. Due to some flaw in the internal design, timeout cannot be used in this case. Regards, Michal > > On Wed, Oct 17, 2018 at 5:09 PM Michal Va?ko wrote: > > > Hi Vladimir, > > > > On Wednesday, October 17, 2018 10:37 CEST, Vladimir Georgiev < > > vladimir.georgiev at telco.com> wrote: > > > > > Hi, > > > I am writing to you, because we have problem using sysrepo. > > > We are trying to use sysrepo as a database for machine configuration. > > > We wrote several subscribers on Python that are services, started via > > > systemd. During their start they subscribe itself to sysrepo. > > > When all services/subscribers are subscribed and when there are some > > > > > changes in sysrepo, all corresponding subscribers are called, as expected > > > to be. > > > > > > The problem become visible on the next restart. The subscribers are > > unable > > > to subscribe itself. Every request for subscribing raise exception > > > "Sysrepo-internal error" > > > After some investigations we saw that the persist file (in > > > /etc/sysrepo/data) that is for corresponding root yang file, where is > > > stored data of subscribers, socket, priority and etc. is broken (It is > > > filled partially). > > > It become in broken state during restart. The only fix, that we found for > > > the situation, is removal of that file. But that is untill next changes > > in > > > sysrepo and calling of subscribers, so after restart. > > > > We have not experienced anything like you described. My first advice is to > > update sysrepo and libyang to most up-to-date versions, it would be > > difficult to help you further without that. Now, what do you mean by > > restart, you simply terminate sysrepod and execute it again? You checked > > the persist file right after sysrepod restarted and it was corrupted? Was > > it already corrupted after sysrepod was terminated? > > > > > I presume that the problem could be connected with simultaneous starting > > of > > > services that lead to simultaneous subscribing requests/calls. > > > > Perhaps, but each of these persist files should be file-system-locked for > > this not to be an issue. > > > > > Another thing is that the persist file is created with permissions 644 > > (all > > > others persist files are 666), on the same time in sysrepo corresponding > > > yang file is imported with permissions 666 (I do not know is there is a > > > connection between both, but I do that remark, just in case) > > > > That is definitely suspicious and you should try to find out how is this > > file created if you can. sysrepoctl should set 0666 permissions on all the > > files it creates. > > > > I am not able to help you more, maybe someone else from sysrepo members > > can? Maybe the python bindings are to blame? > > > > Regards, > > Michal > > From mvasko at cesnet.cz Thu Oct 25 13:57:49 2018 From: mvasko at cesnet.cz (=?utf-8?q?Michal_Va=C5=A1ko?=) Date: Thu, 25 Oct 2018 15:57:49 +0200 Subject: [sysrepo-devel] new sysrepo limitation Message-ID: <5573-5bd1cc00-29-56f5b600@198661279> Hello, during implementation of the new sysrepo we have encountered a problem. We could come up with only either too complicated or too limiting solutions so I would like to present the one finally chosen as the most suitable one considering every aspect. If you have serious problems with it, please speak up now. Generally, it is difficult to change libyang context (loaded models and their features) because we are trying to use it whenever possible to be efficient. To explain a bit more, if there are some internal data connected to the current context, updating the context and the internal data has proven to be too complex and ineffective to be the solution. We are proposing that it would be possible to remove models and/or change enabled features only during the main shared memory creation, in other words, when the first client connects after a reboot. If you remove models/change features after that, it would just be stored in a persistent storage and applied after the next reboot. Note that adding new models with a specific set of enabled features would be allowed anytime. Kind regards, Michal