[sysrepo-devel] ?==?utf-8?q? Issues with sysrepo's persist file

[Michal Vaško]

Hi,
sorry to hear that but I do not think we will be of much help.

On Tuesday, October 23, 2018 13:03 CEST, Vladimir Georgiev <vladimir.georgiev at telco.com> wrote: 
 
> Hi again,
> 
> I want to thank you for your reply. Unfortunately we continue our saga with
> subscribers.
> 
> We were able to isolate strange situation. There are no subscribers to
> sysrepo. Sysrepod started with debug options (-d -l 4).
> During execution of "sysrepocfg  --get /ucpe:config//* -f json", we got
> error that there are no subscribers (that is as expected), but on sysrepod
> logging we are seeing strange WRN message:
> 
> [DBG] New message of size 29 bytes received.
> [DBG] Processing session_stop request (session id=96703980).
> [DBG] RP session stop, session id=96703980.
> *[WRN] There are some (1) unprocessed messages for the session id=96703980
> when session stop has been requested, this can lead to unspecified behavior
> - check RP caller code!!!*
> [DBG] Sending 33 bytes of data.
> [DBG] 33 bytes of data sent.
> [INF] Dropping session id=96703980.
> [DBG] fd 6 readable (revents 1)
> [DBG] Peer on fd 6 disconnected.
> [INF] Closing the connection 0x5555557d6df0.
> 
> 
> We got a lot of these *WRN* messages during the subscribing of our
> subscribers to sysrepo (when sysrepod is active), but recently we were able
> to isolate such message, without our subscribers. Only sysrepod and
> sysrepocfg
> 
> Is that behave is normal?

No, whenever you see that message, something is seriously wrong. If this always happens to you right after starting sysrepod, there is some persistent internal problem so I would suggest to remove the whole repository and reinstall sysrepo.

> Also I want to ask few other things:
> - At the moment we create a connection, then create session then do
> subscribe the subscriber. But both connection and session are open till
> stopping of the subscriber from systemd. Is that is the right way, or
> session should be closed and opened only when it is needed?

It is fine this way.

> - Do you have any idea why when we try to make connection and then session
> from another subscriber (one of both functions throw exception with
> sysrepo-internal error) . Like resource is locked. Now we have retry
> mechanism, but it is like brute force - try until succeed, but that is not
> elegant How to recognize when resource is free, so only then to do try to
> subscribe?

Unfortunately, there is no better way. Due to some flaw in the internal design, timeout cannot be used in this case.

Regards,
Michal

> 
> On Wed, Oct 17, 2018 at 5:09 PM Michal Vaško <mvasko at cesnet.cz> wrote:
> 
> > Hi Vladimir,
> >
> > On Wednesday, October 17, 2018 10:37 CEST, Vladimir Georgiev <
> > vladimir.georgiev at telco.com> wrote:
> >
> > > Hi,
> > > I am writing to you, because we have problem using sysrepo.
> > > We are trying to use sysrepo as a database for machine configuration.
> > > We wrote several subscribers on Python that are services, started via
> > > systemd. During their start they subscribe itself to sysrepo.
> > > When all services/subscribers are subscribed and when there are some
> >
> > > changes in sysrepo, all corresponding subscribers are called, as expected
> > > to be.
> > >
> > > The problem become visible on the next restart. The subscribers are
> > unable
> > > to subscribe itself. Every request for subscribing raise exception
> > > "Sysrepo-internal error"
> > > After some investigations we saw that the persist file (in
> > > /etc/sysrepo/data) that is for corresponding root yang file, where is
> > > stored data of subscribers, socket, priority and etc. is broken (It is
> > > filled partially).
> > > It become in broken state during restart. The only fix, that we found for
> > > the situation, is removal of that file. But that is untill next changes
> > in
> > > sysrepo and calling of subscribers, so after restart.
> >
> > We have not experienced anything like you described. My first advice is to
> > update sysrepo and libyang to most up-to-date versions, it would be
> > difficult to help you further without that. Now, what do you mean by
> > restart, you simply terminate sysrepod and execute it again? You checked
> > the persist file right after sysrepod restarted and it was corrupted? Was
> > it already corrupted after sysrepod was terminated?
> >
> > > I presume that the problem could be connected with simultaneous starting
> > of
> > > services that lead to simultaneous subscribing requests/calls.
> >
> > Perhaps, but each of these persist files should be file-system-locked for
> > this not to be an issue.
> >
> > > Another thing is that the persist file is created with permissions 644
> > (all
> > > others persist files are 666), on the same time in sysrepo corresponding
> > > yang file is imported with permissions 666 (I do not know is there is a
> > > connection between both, but I do that remark, just in case)
> >
> > That is definitely suspicious and you should try to find out how is this
> > file created if you can. sysrepoctl should set 0666 permissions on all the
> > files it creates.
> >
> > I am not able to help you more, maybe someone else from sysrepo members
> > can? Maybe the python bindings are to blame?
> >
> > Regards,
> > Michal
> >