Professional OPC
Development Tools

logos

Online Forums

Technical support is provided through Support Forums below. Anybody can view them; you need to Register/Login to our site (see links in upper right corner) in order to Post questions. You do not have to be a licensed user of our product.

Please read Rules for forum posts before reporting your issue or asking a question. OPC Labs team is actively monitoring the forums, and replies as soon as possible. Various technical information can also be found in our Knowledge Base. For your convenience, we have also assembled a Frequently Asked Questions page.

Do not use the Contact page for technical issues.

Handling of PubSub messages with custom data types inside

More
26 Jul 2023 05:36 #11956 by H64P
Hello.
Sorry that I am just now getting in touch with you. I spoke to our it experts. They are working on a solution to make the device with server available for you. I keep you updated.

Best regards.

PS: Because of privacy concerns we should not working public on a solution, but the the findings with not sensitive data we should share in that thread after we completed.

Please Log in or Create an account to join the conversation.

More
21 Jul 2023 13:00 #11943 by support
Hello,
thanks, and I am glad that disabling the plugin helps as a workaround.

Everything looks good to me in your server now. Troubleshooting this "off-line" gets difficult at this point. Would it be possible that you either deliver your server/configuration to us so that we can reproduce the issue here, or that you expose the server endpoint to us (through firewall from a dedicated source IP)? Also possible would be if you can export the PubSub configuration into .bin file (not sure if open62541 supports it) (but then it's not assured that it will also reproduce the problem) plus possibly capture the PubSub packet through Wireshark into a pcap file. If none of this is possible I will still find some ways, but they would be much more demanding and the progress slow.

We can continue the work on this over email if you are concerned about privacy - in such case, please email to support09 (at) opclabs.com.

Best regards

Please Log in or Create an account to join the conversation.

More
21 Jul 2023 10:31 #11942 by H64P
Hello and thanks for your offer with the research. Let's get on to work.

For debugging purposes we have reduce the pubsub message from three to one field.We added the StructureDataTypes of this field to our server and I can find it in UAExpert as you described:


The message I'm getting is still the same:
"Data type Id not found for encoding Id: ns=2;i=15016 (available encoding Ids: 0, available data type Ids: 0)."

Do you need further informations for your research?

Thanks for the hint with the plugin disabling. That's it. Now I'm getting the byte array of the payload for manual data decoding. Good interim.

Best regards.
Attachments:

Please Log in or Create an account to join the conversation.

More
20 Jul 2023 13:18 #11938 by support
Hello.
I am definitely interested in making this work, if your server is doing it right now. I really am - we may find some issue on our side, or at least improve the diagnostics.

1. So, you can now find StructureDataTypes in UaExpert at the place I described, right?

2. Is the message you are getting still the same? Does it still indicate zero numbers of data types/encodings ("...(available encoding Ids: 0, available data type Ids: 0)"?.

3. For own decoding, please try this after instantiating the EasyUASubscriber:

 
subscriber.InstanceParameters.PluginSetups.FindName("UAComplexDataSubscriber").Enabled = false;
 

I could not easily try it out now, but the aim is that you would get your normal fields already decoded from us, but for fields that have the custom data you would get the UAExtensionObject, from which the raw byte array *of just that field* can be obtained, and "manually" decoded.

Best regards

Please Log in or Create an account to join the conversation.

More
20 Jul 2023 12:08 #11937 by H64P
Hello and thank you for your reply. I'm not sure if I understood that correctly, so i must ask again. Thanks for you patience.

Our server produces pubsub messages (pseudo-code):
<Variant>
  <CustomDataType>
    <UInt/>
    <FloatArray/>
  </CustomDataType>
  <CustomDataType>
    <Int/>
    <ByteArray/></CustomDataType>
  <CustomDataType>
    <String/>
    <Byte>
  </CustomDataType>
</Variant>
In the past I used
#region Get metadata
var publishSubscribeClient = new EasyUAPublishSubscribeClient();
IUAReadOnlyPubSubConfiguration pubSubConfiguration = publishSubscribeClient.AccessReadOnlyConfiguration("opc.tcp://10.42.29.58:4840");
var pubSubConnectionNames = pubSubConfiguration.FindPublishedDataSetElement("<datasetName>", null);
var metaData = pubSubConnectionNames.DataSetMetaData;
#endregion Get metadata
to retrieve the metadata for the variant, but seemingly not for the nested custom data type.
I've checked the StructureDataTypes as you described, no items exists.
Therefore we added the custom data type to the StructureDataTypes at our server. Now the custom data types are items of StructureDataTypes.
But unfortunaly this does not fix our issues. Do you have further tipps/approches or are you interested in further informations for internal researches?

A further question:
Can we retrieve the data ("payload") when EasyUASubscriber generates an error for "conversion failed", so that we can decode it by hand?
Manually decoding is not a nice job but it would be okay.
(At the moment i have written my own pubsub client for a special appliance, but the byte parsing to get the payload is not a pleasant job and my solution not generic enough.
So it would be nice to get the data from you and I do only the conversion of the payload manually.)

Many many thanks for your support.

Best regards.

Please Log in or Create an account to join the conversation.

More
18 Jul 2023 11:57 #11923 by support
Hello.

Unfortunately this is not possible, even as workaround.
Note that in general case it cannot be done at all, because the data types can have subtypes. So, if e.g. the data type of the field in the dataset is declared as BaseDataType, it can be anything. And that would require obtaining the whole data type tree - which can be also dynamic, and very large. That is one of the reason why the descriptions for all concrete possible datatypes must be explicitly listed directly in the published dataset metadata - so that the subscriber can prepare itself for the finite and static set of data types.

Some publishers allow to export the configuration into a binary file - and QuickOPC can read it - but since it is essentially the same structure, I doubt that if your publisher supports it, that the data types will be there,

In essence, the server (or open62541) needs to be fixed.

Best regards

Please Log in or Create an account to join the conversation.

More
18 Jul 2023 09:56 #11922 by H64P
Thank you for your reply.
I've checked the DataSetMetaData as you described: StructureDataTypes -> StructureDescription Array[0]
So there are no descriptions.

Is there a way to use QuickOPC to get the metadata for our custom datatypes from the client-server model and provide that to the handler for PubSub? Or do we have to manually extend our server (implemented with open62541) with it?

At this point, many thanks for your great support!
Best regards.

Please Log in or Create an account to join the conversation.

More
17 Jul 2023 17:43 #11917 by support
Thank you,
this has helped. I will for now assume that the error message was just imprecisely reported, because it otherwise *is* the one I am looking at - the Error ID does match.

It is good to know that you can see the custom data type in the address space of the server (under Types -> Data Types). Unfortunately, it is *not* where the subscribers look for it. For PubSub, the type definition must be contained directly in the metadata for the published dataset. And I suspect this is not the case with your server, because the inner exception says "... (available encoding Ids: 0, available data type Ids: 0)".

To check this in UAExpert:

1. Navigate to this node: Root -> Server -> PublishSubscribe -> PublishedDataSets -> yourDatasetName -> DataSetMetaData.
2. Drag and drop this node into the Data Access View.
3. In Data Access View, double click in the "Value" column for the node row. It will display the "Edit Value" property grid window.
4. Look at the "StructureDataTypes" row. How many StructureDescription elements are there? Zero? Or more?

Note: Subscribers cannot use the data type definitions from the "usual" place (in client-server model), mainly because PubSub must be able to work in a disconnected manner. And just from the types in the address space, the subscriber does not know *which* data types are relevant and can ever appear on the wire in the actual PubSub messages. But the subscriber needs to have this information upfront - it cannot interrogate the server when it receives some data over PubSub - that would require making a connection over client-server. So the subscriber must have this information either using PubSub discovery messages from the server, or using the DataSetMetaData for the published dataset - and in it, the server "summarizes" the custom data types that can appear in the data, in a self-contained, self-describing way.

Best regards
The following user(s) said Thank You: H64P

Please Log in or Create an account to join the conversation.

More
17 Jul 2023 11:10 #11916 by H64P
Hello and sorry for the mess. I will make it clear step by step.

1.) I noticed that the link in the initial post no longer works and I can't set a working link today. Therefore here is the headline of the example I've used "Examples - OPC UA PubSub - Extract a field from the message" refer to the documentation.

2.) I copied the error message in the intial post from the terminal while testing some examples. I'm sorry but I can't reproduce the error message today. So please ignore it.

3.) Here is the source of my software under test, it's a modded/extended example from your documentation:
using OpcLabs.EasyOpc.UA.PubSub;
using OpcLabs.EasyOpc.UA.PubSub.Configuration;
using OpcLabs.EasyOpc.UA.PubSub.InformationModel;
using OpcLabs.EasyOpc.UA.PubSub.OperationModel;
 
#region Get metadata
var publishSubscribeClient = new EasyUAPublishSubscribeClient();
IUAReadOnlyPubSubConfiguration pubSubConfiguration = publishSubscribeClient.AccessReadOnlyConfiguration("opc.tcp://10.42.29.58:4840");
var pubSubConnectionNames = pubSubConfiguration.FindPublishedDataSetElement("<datasetName>", null);
var metaData = pubSubConnectionNames.DataSetMetaData;
#endregion Get metadata
 
UAPubSubConnectionDescriptor pubSubConnectionDescriptor = "opc.udp://224.0.0.22:4840";
pubSubConnectionDescriptor.ResourceAddress.InterfaceName = "<myAdapter>";
var filter = new UASubscribeDataSetFilter(UAPublisherId.CreateUInt16(390), writerGroupDescriptor: 100);
 
// Instantiate the subscriber object and hook events.
var subscriber = new EasyUASubscriber();
subscriber.DataSetMessage += subscriber_DataSetMessage_ExtractField;
 
Console.WriteLine("Subscribing...");
subscriber.SubscribeDataSet(pubSubConnectionDescriptor, filter, metaData);
Console.WriteLine("Processing dataset message events for 20 seconds...");
Thread.Sleep(20 * 1000);
Console.WriteLine("Unsubscribing...");
subscriber.UnsubscribeAllDataSets();
 
Console.WriteLine("Waiting for 1 second...");
// Unsubscribe operation is asynchronous, messages may still come for a short while.
Thread.Sleep(1 * 1000);
Console.WriteLine("Finished.");
 
static void subscriber_DataSetMessage_ExtractField(object sender, EasyUADataSetMessageEventArgs e)
{
    // Display the dataset.
    if (e.Succeeded)
    {
        // An event with null DataSetData just indicates a successful connection.
        if (!(e.DataSetData is null))
        {
            // // Extract field data, looking up the field by its name. UADataSetFieldData
            // int32FastValueData = e.DataSetData.FieldDataDictionary["Int32Fast"]; Console.WriteLine(int32FastValueData);
        }
    }
    else
    {
        Console.WriteLine($"*** Failure: {e.ErrorMessageBrief}");
    }
}

The metadata I received contains three fields with the expected field names. Data type of field[0] has i=15015 and its referenced encoding is i=15016. I have this information from design time and can also get/resolve this data with UAExpert. But the software under test throws an exception. For details refer to the attached screenshot.

Is the software under test okay or is something missing or wrong? Does QuickOPC get the DefaultEncodingID automatically or do I have to trigger it?

Best regards.
Attachments:

Please Log in or Create an account to join the conversation.

More
17 Jul 2023 09:19 #11914 by support
Hello.

I find it difficult, almost impossible to believe that the error message you reported,

Data conversion failed for record field '<field name>' (value type name 'UAExtensionObject').

comes from QuickOPC 5.71.273, or, in fact, from any version of QuickOPC. The message format we use is:

$"Dataset field '{failedField}' data conversion failed (value type name '{failedFieldData.ValueTypeName}')."


The initial part seems to be missing, and the remainder slightly different, in your report. This may seem like nitpicking, but without being clear on the symptoms of the problem, and being sure about the software under test, providing further advise makes little sense.

Can you please recheck.

Best regards

Please Log in or Create an account to join the conversation.

Moderators: support
Time to create page: 0.073 seconds