Professional OPC
Development Tools

logos

Online Forums

Technical support is provided through Support Forums below. Anybody can view them; you need to Register/Login to our site (see links in upper right corner) in order to Post questions. You do not have to be a licensed user of our product.

Please read Rules for forum posts before reporting your issue or asking a question. OPC Labs team is actively monitoring the forums, and replies as soon as possible. Various technical information can also be found in our Knowledge Base. For your convenience, we have also assembled a Frequently Asked Questions page.

Do not use the Contact page for technical issues.

Subscription mode. Catastrophic failure error -> blocked Subscribe/Unsubscribe

More
22 Jan 2020 10:36 #8161 by Andriy
Hello,

My application accesses to 3 OPC DA servers and 1-2 OPC XML-DA servers.
Do you have any progress?

Best regards

Please Log in or Create an account to join the conversation.

More
20 Jan 2020 17:17 #8158 by support
Hello,

are you mixing in your application access to OPC DA and OPC XML-DA servers?
Not that there is anything wrong with it. I just noticed it in the call stacks, and want to verify whether it is intentional.

Regards

Please Log in or Create an account to join the conversation.

More
16 Jan 2020 08:30 #8137 by support
Thank you - that's what I needed.
I will analyze it and let you know here.

Best regards

Please Log in or Create an account to join the conversation.

More
14 Jan 2020 10:52 #8133 by Andriy
Hello,

Thank you for such a detailed answer.
I prepared 2 files
  • file threads_short.txt contains the short list of all opclabs related threads
  • file threads_callstack.txt contains all opclabs related threads with expanded call stacks. I filtered out my application's instructions

Let me know if you need more information. This issue is very critical to me
Attachments:

Please Log in or Create an account to join the conversation.

More
11 Jan 2020 12:59 #8131 by support
Hello,

there are various reasons for the "Catastrophic failure" errors. One that I have seen relatively often is that the COM infrastructure transforms unhandled native code exceptions (such as when the server is written in C/C++ and throws an exception during a COM method call) to "Catastrophic failure". So in this sense yes, the very origina of it *can* be in the OPC server. In such case the expected behavior of QuickOPC would be to try to reconnect after a configured period.

But, there is also a possibility that the "Catastrophic failure" comes from a QuickOPC bug. That would be an internal error, and in such case, anything that happens *after* the first error is undefined. It can even, indirectly, cause a deadlock.

So, we need to explore all these options. I'd like to first look at the deadlock. The screenshot and the callstacks you have provided are nice, but do not give me enough. Specifically, in the callstacks.txt, I would need more threads expanded: Not necessarily all of those you have on the screenshot, but one of every kind: I mean, if you look at how they are displayed on the screenshot, each has the location where it hangs - but many of them are the same - and the call stacks are most likely the same as well. So, can you please provide the remaining call stacks?

Here is what I expect to get from the analysis - one of two outcomes:
a) If the threads are blocked, and it can be seen from the call stacks that there is a combination of locks causing the hang, it would be quite clear that this needs to be avoided in the QuickOPC code, so we will look at it.
b) If the threads are blocked but no obvious reason can be seen from the call stacks: In such case it is possible that a lock has remained entered, but the thread that did it is gone (aborted). That would be an indication of a problem in QuickOPC but elsewhere - not sure where.

Many thanks

Please Log in or Create an account to join the conversation.

More
10 Jan 2020 10:04 #8129 by Andriy
Hello,
Do you have any update?

Please Log in or Create an account to join the conversation.

More
08 Jan 2020 13:15 #8121 by Andriy
Hello,
Sorry for the long delay in replying.

Firstly, answering your questions
1. I'm referring to a contents of the Exception property of the event arguments object
2. The contents of the Source and StackTrace properties of the exception object: Source: "OpcLabs.EasyOpcRaw.DataAccess.RawEasyDAClient", Stack trace: null
3. Inner Exception is null
4. I rely on QuickOPC ability in order to reconnect to OPC server.
5. The sequence of events is the next
  • Subscribe to opc items
  • Sometimes I get data succesfully, sometimes I receive receive the "Catastrophic failure" in the Exception property immediately
  • Than, more often I get the following errors. The order of these errors is undefined:
    System.Runtime.InteropServices.COMException (0x8007046A): Not enough server storage is available to process this command.
    System.Runtime.InteropServices.COMException (0x80070008): Not enough storage is available to process this command.
    System.Runtime.InteropServices.COMException (0x800706BF): The remote procedure call failed and did not execute.
    System.Runtime.InteropServices.COMException (0x800700A4): No more threads can be created in the system.
    System.Runtime.InteropServices.COMException (0x8000FFFF): Catastrophic failure
    System.Runtime.InteropServices.COMException (0x80004005): Unspecified error

6. I'm connecting to both OPC servers: local and remote. I get exceptions from both
7. I confirm that this referred to the already running instance of my application. I also confirm that other OPC clients and newly started instance of my application can connect to it

I decided that this is a OPC server problem because it happens when communicating to both servers: local and remote. Therefore, I added my own "no-comm" detection logic that do the following steps when "no-comm/lost connection" is detected:
  • Unsubscribe from all opc tags
  • Wait 5 min
  • Subscribe to opc tags again
I did't create new instance of EasyDAClient in this case because I found in the opclabs forum that it can result in unmanaged resources leaking.
Unfortunately, I get another problem: opclibrary deadlock.
I make 2 application memory dumps with 30 min delay. Near 15 threads are freezing at the same instruction: trying either subscribe or unsubscribe the same opc items (see attachments)

I don't know how to fix this issue. Can you provide opclibrary symbols files?
Product version: 5.55.433.1
Language: C#
Attachments:

Please Log in or Create an account to join the conversation.

More
01 Nov 2019 07:03 #7905 by support
Hello,
I have some additional questions.

1. I assume you are referring to a contents of the Exception property of the event arguments object you receive - please confirm.

2. Can you please provide the contents of the Source and StackTrace properties of the exception object.

3. If there is a non-null InnerException (probably not), please provide details about it as well.

4. Are you doing/trying to do something "special" in order to reconnect, or you rely on built-in QuickOPC ability (which should normally work by itself, I agree)?

5. Can you please describe precisely the sequence of of events. For example, you subscribe, you get data successfuly, then at some random moment (related to server crash?), you receive the "Catastrophic failure" in the Exception property of the event arguments. What happens then (for the same OPC item)? Do more errors come - the same ones, or do you ever get a different error? Or do you receive a single "Catastrophic failure" error, and from that moment on, nothing else for the same item?

6. Which target OPC server are you connecting to? Is it local or remote?

7. You wrote "my application after that cannot reconnect to the OPC server when it is up and running". Please confirm that this referred to the already running instance of your application. And, after the server crashed and is running again, can other OPC clients connect to it, and can a 2nd, newly started instance of your application connect to it?

Thank you

Please Log in or Create an account to join the conversation.

More
30 Oct 2019 16:47 #7901 by Andriy
Hello,

I receive a lot of "Catastrophic failure" errors on ItemChanged event handler.
I don't know what it means and how I can handle it.
Can you provide information about this error: when it is risen and how I can fix it.
I don't know if it's connected or not, but I get this error when the OPC DA server periodically crashes.
What is worse, my application after that cannot reconnect to the OPC server when it is up and running

QuickOpc version=5.54.1115

Thank you

Please Log in or Create an account to join the conversation.

Moderators: support
Time to create page: 0.065 seconds