Browser integrations: using hardware and access to the filesystem

We’re in number 30 of our ongoing series every two weeks. Last week we did an update on a new available web packages. And this week I’m kind of excited to show you something we’ve been working on towards the end of last year for doing browser to desktop integrations. So I’ll just jump right into it. I do have some demos, but just a quick overview of why we’re doing this before I do those. So traditionally from a browser, you can’t get to resources on the desktop unless you use some proprietary plugins like ActiveX controls or something, even those are quite limited and difficult to use and aren’t supported across many browsers. So this has always been sort of a topic that comes up when we’re looking at requirements for an application that has to run into browser. Yet it needs access to things like the local or network file system, printer scanners, you know, controlling ports and hardware, cash drawers, things like that, scanners, barcode scanners, and also other installed software too. So to be able to integrate with maybe a word or Excel or any other installed software. So this has typically been a blocker, but recently we did some research into this to see, you know, we think we could probably figure out a way around this or a couple ways around this. So we did start a new project and we looked into bypassing this limitation. And the way we did is because the release of the client manager plugin and Servoy 8.2 included a broadcast mechanism, we’re able to use Servoy smart client and broadcast between a browser and, you know, an agent that will serve as the desktop agent and allow you to do desktop things from the HTML5 client. So we created a project called SUI desktop agent and we trialed this with several customers so far, quite successfully, in proof of concepts where we did things like we called network scanners and we obtained images, we printed to network printers, we did calm integrations with Microsoft Word and Excel, I believe. So, and we talked to the file system directly. So once you get over that bridge, you can pretty much do anything that a desktop piece of software can do. So let’s have a look with some demos. And what I’m going to do is I’m going to show you how to, how, in this, you know, in a simple case, how we could interact with file system, how we could open a file with a native application and how we could also monitor the file system for changes. You can see that I have running in my browser a little explorer here that I’m looking at my local file system. And this doesn’t use any plugins or anything. It’s actually talking through the broadcaster. I’m going to expand and go to, well, I have a little bug there where it collapses the first time. But I’m going to go in and I keep drilling into, let’s go, I don’t know, look at a log file or a properties file, something like that. So every time I drill in here, it’s actually requesting information from the desktop agent. And then if I select, let’s get a properties file here. You can see I’ve recovered some of the information, the file name, the path, last modified size. And what I’d like to do is open this in the default editor. So typically in a browser, you’d have no access to be able to see what software there is, let alone open a file, and in a certain software. So I’m going to open this in the default editor, which is Notepad++. You can see that it launched it over here in my editor. Now the next thing I want to do is to be able to monitor that file. So I’m going to come back to the browser here and I’m going to click the watch button. And what I’ve done is I’ve added an event listener in the desktop to know when that file has been modified. So if I come back here and do a Hello World of sorts. And when I save, I should get a callback. And you could see that I refreshed, there was a little flicker there. I refreshed the file. And you can see the timestamp for last modified now matches the clock on my system. So every time that I save that file or it could be, you know, delete file in that directory, whatever, I’m getting a callback back in the browser client to know that that file has been changed. And in this case, I just updated the information about it in the timestamp. So that’s really it. Those are the three parts of the demo. Number one is just seeing the file system. Number two was opening a piece of software on the desktop with the select file. And the third one was how to watch a file for changes. I can take you in under the hood a bit to show you what goes into that. Let’s start with the project that we’re using for this. So I have an application over here called example desktop agent. And you can see that it has a module included called the SUI desktop agent. And there’s a base form here called abstract agent. And all you have to do is extend this base form as a way of exposing some functionality that you want to be able to call remotely from, say, a browser. So what I’ve done in my project is I’ve created a provider called test agent provider. And it just implements a couple methods. One is to get an action, which is really a function based on a name. So first it can expose what action names are available. And the other one is, okay, so I get an action command string and I want to be able to call that remotely. And so then this just returns one of these two functions to get file info in the open file. The other part of this are the events. So there’s a method here to get event names. And you can see that there’s only one here I have supported in the enum for on file change. Then there is another method that you can override from the base form, which is how to register for an event. And this is something that you would do, you would handle in your own custom logic. But it does pass in the event name and any optional extra information that you want. So in this case, the only event we have is that on file change and the piece of extra information that we passed in is just the file path that we want to watch. And so we have this method called watch file, which is our own custom logic, which just starts watching that file. I don’t think that it’s worth focusing on that. That’s the custom part, not really just scaffolding that sets up how this works. If we look now at what’s calling this, we have this form called test agent. I’ll open this in the designer so you can. Open it for a matter. So you can see that on the. On the left we have that tree component, which shows the file and on the right we have the details. And this is what we see in the browser and when we want to refresh that tree, we’re making that remote call. So the way this works is I have some. I’ll open this in the script editor now. And I have some methods in here. One is to list the files. And what we do is we pass in the directory that we want to start from. And it’s calling the svi desktop agent project and it just does this call method and it passes in a string, which is that action command, which references that enum in our provider. Some information. And a callback. So this information could be anything. It’s just optional. In this case, it’s just the path that we want to. And then a callback because this is a synchronous. It’s just a broadcast. And that’s going to receive a broadcast. It’s not. It’s not synchronized. So then we go into our on list files, which is the function down here, which then deals with capturing the info and refreshes the tree. That part isn’t really worth focusing on. It’s the scaffolding that. That allows that to happen, which is saying I’m going to call this remote action. I’m going to pass in some custom information and I’m going to receive a callback when it’s done. The other part of this was when we went to register a watcher. I’m sorry, first, I’m getting out of myself to open the file. So I have this other method down here called open file. And it’s the same thing. We’re doing that. SUI desktop agent API call. We’re passing in the open file command, which is just a string. And the custom information here is. Is the path that we want to open or the node that we’re getting that’s selected in. And the tree. And that’s what that’s what opens it in. On the desktop side. So the desktop agent project handles all of the all of the plumbing it handles. serializing this stuff and broadcasting it as a message, receiving it, figuring out what to do with it. And mapping it to the right target action that gets called. The third example we did was to watch the file. So we have this method down here watch file. And in this case, instead of doing the call method of the SUI desktop agent, we’re doing the register. In this case, we’re passing in again a string. It’s not an action name this time. It’s an event name and a callback function that happens back here in the browser or in the client session for the browser and then some extra info that could be anything. In that case, this is just the path that we want to watch. We’re registering a listener for this and then we’re registering the handler that gets called when that event happens down on the desktop. So that handler is back here. On file change, I just print something out to the console and I refresh the file which is just a custom logic here that gets the file info and updates the tree and the fields that are showing on the form. But it’s really this SUI desktop agent project that we have here that allows us to do that. It’s pretty simple if I show it to you and ask for code complete. You’ll see that there’s really only a few methods. There’s the call method which we do from the side of the browser client or the register method. Then the invoke is happening from the desktop side and we can take a look at that. So if we go back to our provider, when there’s an event, I’m calling this on file watch when there’s an event on the file system that I’m watching for. I’m calling that invoke method again on that SUI desktop agent project and I’m saying the event name is this on file change and then there’s some custom info. And what the desktop agent project will do is it’ll route that response or that notification that an event happened, it’ll route it back to the right callback method back in the ng client session so that it gets called. So in this way we knew that the file had been modified and we were able to update the timestamp. So that’s the demo and that’s looking under the hood. Let me go back to our slides here and just give you a recap. So the setup is, I’ll put this in full screen mode. The setup here is that the abstract agent is that base form which just exposes a few methods and that’s what you, that’s what you use to discover implementations of what we call agent providers and so all you have to do is extend that base form and you expose your callable actions and your observable events that, you know, these will happen in the desktop and then you use the API to call those actions for to register those events and you’ll receive those callbacks back in the ng client session and you can run your logic there. The agent provider will expose those actions by implementing a couple methods. There’s a typo there on get action names that shouldn’t have two S’s there. But you, you know, here are the action names and here’s the action which maps to a function that we can call. And then on the HTML5 client, the ng client session side, we use that API to call it and handle the callbacks. We use that call method. We pass in this on list files and then that gets called when the desktop is done doing its thing. The agent provider, again, down in the desktop, it exposes observable events. In this case, there was just one that on file change and then back on the ng client session, we can register for those events and that calls this register method which you run some custom logic. This is down in the desktop. This is what, you know, added a watcher to the directory to or the file to get the call back. In the HTML5, the ng client session, we have this, the simple API you just call register. You give the event name, you give it the call back and some extra info and then we handle that in the callback that on file change and that’s where we refresh the timestamp. On the agent provider side down in the desktop, when we want to call an event, there’s just a simple API to call and invoke. You pass in the event name and then some custom information. So that’s pretty much how it works. I should point out that it does require the smart client to be running, which means you have to be able to run Java on the machine that you want to talk to. And the smart client has to be running, but it can also, there’s a way to launch it with the deep link. So if it’s not already running, if you don’t already have a running agent for the ng client session, you can launch it through a deep link. Although that part, the very first time it launches, there will be like a delay when it’s downloading so it wouldn’t be a seamless experience you did until you got that agent running. So there are, I think there’s some kinks to be worked out there in terms of user experience. But from an architecture point of view, once the smart client is running, you have a pretty good proxy to the desktop. Again, we’ve worked on this with a couple of customers already in proof of concepts, but right now it’s in a project on GitHub, which is still currently a private repository. So if you’re interested in trying this out or you have some use cases that you want to run past us, what I’m asking is that you contact us directly and we’ll take a look at your use case and see how we can apply this to that, if it’s a good fit. I’ll post the slides and link to the recording on the forum and you can always visit all of the prior recorded sessions on our website at the Tech Series page. So that’s it. Y’all, do we have any questions? Oh, there was a nice and brief session. Yeah, there’s a couple of questions coming in. All of them is what operating systems can you use for the desktop agent? Right. Well, because it is just using a smart client, that’s cross-platform, and it’s Java-based, so we can support any operating system that runs Java. Okay. So your client, so the client could be any client within a network running Mac Windows and Linux, correct? Or you may potentially even the server? Yeah, that’s correct. Okay, excellent. I think in one use case, we used a desktop computer with a connected scanner to scan documents is that right? Yeah, yeah, we talked to a couple of network scanners to scan documents and we pretty much set up a form, put a button on it, click scan, and then we showed a progress bar, and then a few seconds later the image showed up right there on the on the forum in the browser. Okay. All right, well, let’s so far the only questions I am seeing coming in. I’ll write a few minutes or maybe a half a minute to see if anything else comes in. Any thoughts on the topic two weeks from now, Ashan? Yeah, I mean, bring up, I think I have one more slide. Oh, I didn’t update this slide. Two weeks from now, we’re going to be doing a topic about CSS and seaming. So it’ll be an overview of CSS and the HTML5 client, but also a bit about some research we’ve done into making your applications more themeable so that you can reskin them or re-style them easily. Okay, that sounds pretty cool. All right, well, it looks like this is a very clear topic to everybody because I’m not seeing additional questions come in. So those interested in using this browser to hardware integration, whether it’s scanners or USB drivers or local software, you can integrate all of them right in your browser, feel free to reach out to us and we’ll be in touch with you. We do expect to release this as a standard component, one that has been used a bit more and we’ve figured out the use cases. We probably also have to do some research with regards to security concerns, Sean. I’m not sure how parsing strings from a browser to an executable is protected at this point in time. Well, it’s not called directly from the browser. It’s called from the smart client, so that’s already an authenticated session. So it’s the same as when you run this in the smart client. Okay, well, that sounds good. Well, thanks everybody for attending and we look forward to seeing you two weeks from now. We expect to pose these recordings in a few days on our website. Thank you. Thanks.

Close