Hi,
I’m looking for guidance on how to modify the Theia trace extension so that I can stream data from a target device?
GitHub - eclipse-cdt-cloud/theia-trace-extension: Theia trace viewer extension using the tsp-typescript-client (https://github.com/theia-ide/tsp-typescript-client) and Trace Server Protocol (https://github.com/theia-ide/trace-server-protocol).
Kind regards,
Aske
Hi,
I would like to detail some issues with streaming explaining why trace compass doesn’t support streaming as a standard workflow first, then we I will tell you the easy way to do it. (polling).
Trace Compass does not support streaming because we tried it and the results were user-hostile. When displaying a Gantt chart with nanosecond resolution, updating it every second or so is bad. You really cannot see anything.
If you are looking at x-y plots (line charts/bar charts/scatter charts) the data is also very difficult to follow.
What is needed, in my opinion is to have a triggering mechanism to update the display, which is not available yet. This would lean into the realm of oscilloscopes. Also, we would need to support full redraws on regular intervals.
There was initial work done there. One major issue we faced also is the amount of data to process fills up the network.
A mechanism I can see working, in the medium term is to send a filter to the back-end. Then you query the backend filter, when the data changes, send a full query to update all the views.
I need to be explicit here. With Linux Kernel traces, the amount of data will be huge (GB/S) so performance enhancements on the TSP/Frontend are a must.
We would love to work with you on this, please don’t hesitate to follow up.
Matthew (the tracing nerd)
Hi Matthew,
I’m doing this as part of my university project partnering with Renesas. It isn’t necessarily meant to work as I’m more proving the feasibility making use of stream tracing. I’ve been attempting to make use of chatGPT for guidance. Obviously chatgpt aren’t aware of the issues you listed I’d assume but it still provides me a method for how to add support for stream tracing.
This is what it provided me with:
Is anything Chatgpt said valid?
Hope to hear back from you soon Matthew.
Kind regards,
Aske
Re: ChatGPT. Wow? that’s something. It has many correct words, I disagree with the order of the words though. 
I would like to ask a simple question before going further.
Are you streaming: i.e. sending the trace over the network in chunks?
Or do you want to read the trace live? this is what many people think of when they see the word streaming.
Let’s say it’s streaming over the network. Can you do this outside of trace compass/theia, a simple script can generate the trace then it can be read post-mortem. That would be the simplest way forward.
If you want to do live trace reading, the trace compass server can handle it partially already. Basically you need to define when the live reading has ended (not supported really) and then stop the trace rather than stopping when we reach the end of file. Now again, you will face many “interesting” usability issues, but in this model, you have more levers to control in advance.
I STRONGLY recommend not supporting new even declarations, this will make your job more complex as you will need to update the trace server parsers even more. Assuming you pre-defined all your event types, as I detailed earlier, a polling action would probably be your best bet.
On the server side, you would need to make the trace parser handle “live” data. See CTFTraceGrowingStreamTest as an example of how it could be done. After, you would need to update the views accordingly. On the front end side, this involves polling the back-end to see what the “safe” range to fetch is, and partially updating the views. This is non-trivial work. If you have questions on this, reach out to us.
I hope this answers your question partially at least.
Looking forward to more info.
Matthew
I will be streaming data over a network. Chatgpt provided me with some information regarding a model in the tsp-typescript-client called ‘Stream’. I can’t seem to find it anywhere so I’ve gone ahead and created my own interface.
This is what I have currently:
I’ve created this interface using a sample trace file I’ve been given.
The trace data is from a bus monitor on a board. ( I will upload in another post as it doesn’t let me have a post with 2 images)
This trace data is rather hard to understand. I would recommend first deciding on what needs to be logged, then writing it. If you want to go custom text, maybe this video can help?
But I am under the impression more design work can help.
Question: What is EpochTs? How is it different from epochTS and PMU.EpochTS?
When streaming data, you probably want to keep things as lean as possible.
If the trace cannot be parsed in Trace Compass Classic, the theia front-end won’t work either, so consider tc classic to be a good “incubator”.
Looking forward to seeing some other restults.
Edit: I’m at the reply limit so I am editing this message.
The difference from eclipse trace compass and the theia front-end is pretty simple, though I am not sure it’s documented in one clear place. 
Eclipse Trace Compass, or Trace Compass Classic as I call it, is an eclipse based trace viewer. It also offers a trace server mode. This makes it have no visible UI components. The server follows the Trace Server Protocol, a RESTful API to abstract out all trace visualization back-ends. The protocol is not a server, it is just the description of how a server and client should speak to each other.
The tsp-typescript-client is a typescript implementation (bindings) of the consumer or client for the trace server client. The theia-trace-extension is a typescript based front-end that uses the tsp-typescript-client as a library.
The theia-trace-extension, is the web based front end extension that will be updated in my opinion for polling the data. It already partially supports your use case, see here:
Matthew.
I will get back to you on the trace data, I will be meeting with my manager regarding it tomorrow.
Meanwhile, is there no documentation for the tsp-typescript-client?
Let me know.
Aske
Hi Aske. The tsp-typescript-client is an implementation of the Trace Server Protocol, and the documentation can be found in its swagger page
