Consuming web services in Servoy
Consuming web services in Servoy
Welcome everybody to today’s webinar and today is sort of a part two building on last week’s webinar where we talked about web services today. We go to do the same again but now we’re going to talk about consuming web services and some best practices so with me is are always wonderful presenter. John, welcome to the floor. Good morning, Jan. Good to be here. Yeah, so last week or last session we did how to make your own web service so I think it fits that we also show how to consume web services not just the ones that you make and serve void but maybe maybe other web services. We typically see this I’ve been seeing this a lot that’s why I decided to do this webinar especially with folks who are modernizing modernizing their applications and they have their back end in a legacy system where they want to reuse certain components of that system. We see this a lot with Java dot net progress just to name a few where they might expose some some component like business logic or something through a web service. And so naturally they want to keep their environments in sync so often they’re consuming those web services in server void we also see this for consuming third party services. A lot of times that involves OAuth as a means of authentication we did another separate webinar on that topic a few sessions back so if you are interested in consuming a commercial web service like you know a social web service or anything protected with OAuth. We need to check that out but today it’ll just be a simple plain rest web service example. Of course we’re going to start with demos we have the all important hello world so we did that in the prior webinar where we produce the hello world. And in fact I’m going to be building on the same web service that we that we did before but I’m just going to be consuming it on the other end you might remember that last time. I did it with a with a test tool called postman or you could just see me entering in the URLs and stuff this time we’re going to do it from Servoy. And then we’ll do a comprehensive example where we use the we use that that example database web service. However it’s important to note that a lot of this webinar will be about how to use the in them data source feature of Servoy because if you are consuming a web service you want to maybe have that’s already in the temp table and be able to use all of the things you like about Servoy. I can’t a data bound setup and we’ll talk about best practices and do all the slides in the overview but let’s jump right into the demo. In fact it makes sense if I relaunch this let me do that now. So very first one is hello world so I’m going to put in my name I’m going to push say hello it’s going to send my name to my web service my web service is going to respond with a message that inserts my name in there I could also say hello to my colleague Jan. Very impressive okay let’s jump under the hood real quick and take a look at what’s going on in our simple hello world example. Close all these other than my hello world form. Oops bring that one back okay. So in my hello world form I have that’s not the right one. Hello world client that’s the the server side one. You can see that I have where I enter the name it captures that in a form variable and then I send to my. My service so what I’ve done here is there’s the form variable name. And then I pass that in to I created a top level scope to wrap my service. As a sort of a client API so I have a method called hello world it takes a string it says it’s an API wrapper for the hello world example web service and it returns the message so let’s take a look at. What goes on here so I started with a base URL in this case it’s just local host. Of course that would probably be externalized in a real setup. I add to that the end point and then also the argument name. Then we use the HTTP plug in. I’m calling here get client which is a little utility method I made which is if it’s not already initialized I initialize a client. Which is just a form variable here. And I’m sorry not a form variable a variable on the scope. And so I reuse that client and that’s a that’s a bit about best practice and attack to here number one. It’s sort of a lazy load pattern I’m only going to. Create the client once and I’m going to keep reusing it is a bit more efficient than. Initializing that client every time more importantly the client represents a stateful HTTP session client session so if I wanted to do things like read cookies or. Configure my session timeout or even a HTTP proxy our client proxy I could do that on this object and keep reusing it. So once I have my client here I create a get request I pass in that URL and then the next line I execute the response. It’s important to always check the status code a good service will return an error message and a regular HTTP okay status. And but sometimes you may put in something that doesn’t even get picked up like a 404 or something or or you don’t get authorized at all so you get a 401. So I should probably change this to there’s constants for that HTTP status dot. That status code okay that’s 200 that’s the universal standard for a successful response so be sure to always handle the possibility that you didn’t hit your target. Service return an error message. Finally you want to parse the response body if it’s Jason so you get the response body which gives you a string and then you parse it and you get an object. It’s good to use a type tag because then you can get code complete so down here I may be able to type and get get the message object from that so if you get a complex object. If you put the type tag in you can sort of map it all out and and and use code complete against you know what you know this effect is for the service. So we return that back into the form method and it shows it in a dialogue so that’s the very simple example. Hopefully that’s pretty clear I’m in a build upon that and do a more complex demo which is the data bound example. This looks a lot like the typical orders or details setup however what we’re seeing here is going to be not data bound to directly to back in database. We’re going to pretend that that’s not accessible and we’re data bound to a web service only. So I’m going to go ahead and and push the load button here and it’s going to take a second here to load all of the orders actually not all of them I limited it to. First 50 and and their items and you can see that. I’ve included in this not just the order but also related data like the customer employee and the related items and because I’m using an in memory data source. And I can configure things like calculations so I can even edit say the quantity here and you’ll see the subtotal for the line and also the order total update real time as I tab out so all of that stuff is available on. On in memory data source just like a regular data source. It’s a bit more complex so let’s let’s dive under the hood and have a look. The first thing that happens, let me bring up my. Ws orders form. So when I call this load method. It’s going into again I’ve wrapped everything in in a scope here called web service so it’s doing load orders. What I’ve done is is made a little options object to to handle some of the things that I could pass in for the options for my web service. And you may recall if you watch the last webinar when we produced the web service. We talked about best practices and and doing things like allowing sorting filtering paging. Searching of a record by ID versus getting a list of them back. So I sort of created a little object here to just make a generic request that had options so I could load different endpoints like products customers orders or to details etc. With kind of a reusable pattern so it’s just a little object that holds some of the options that I might send. I’m limiting the page to to 50. I’m also specifying that I’m going to include the related order details again from the last webinar. We showed as a as a best practice to allow the caller of your web service to to specify what additional information they want. So with my orders I also want the child order detail records to go there. So I set up my options. I made it this generic get method because I’m going to load different data sources. In this case it’s the orders end point with those options. And I sort of parse through those options. I don’t think it’s necessary to pay too close attention to this other than you know I use my base URL and my end point. I create a little array of parameters and that’s what I’m pushing those into that array those different options sort include pagination etc. At the end I just append that to the URL query string. Finally I sort of print out what the request is to the debug log and I execute that that request this is a get request. One thing to note is the encode URI here. You want to do that especially when you have you know parameters that you may be adding to the query string that could contain special illegal characters. If you encode that that URL then you’re sure that it won’t it will make a bad request. After you execute the request you can handle the response. So in this case I’m doing HTTP basic authentication so I’m passing in a username and password which I’ve just set up is. Had been admin. This is also following from the example. Web service producing webinar we did where we showed how to set up. Custom authentication on the receiving end. Again remember to handle any non 200 status code. And finally we get the payload. In this case. I’m just printing out the response. I didn’t check for a custom error. But I’ll show an example of that and if I forget to use your message to remind me how to handle custom error coming back from your service. So anyway. I return the object in this case I call Jason not parse so now it’s not a string it’s a complex object. And if we go back to. Where I had called this from. I get I get an object from. From my get now it’s going to look like something like an order where I have properties in it and even nested in there I have the details. And I’m going to call the load. What I can do real quickly is just call. Call the load. Well it’s going to print out a bunch of stuff but I’ll do that. Going to call the load orders again. And you can see. That this is the. This is the response here. It’s just that Jason stuff. And I what I’m doing is just parsing that and converting it into an object. And I’m going to go back to the. Oh here’s where we handle if there’s a custom error. And now what I’m doing is going into the memory data source and loading the memory data source. At this point I’m going to take a invent. And talk about. In memory data sources. So. Again we’re not connecting directly to the normal example database. And I create an in memory data source. If you haven’t done this before. It’s a lot like defining a. A SQL database table in Servoy. Except it’s in fact it really is a SQL database table. Just goes to an in memory. Database implementation. So if I look at my orders. Table here. I’ve I’ve gone in and I’ve added the columns I even set what the pk is going to be. You’ll notice I didn’t include all of your familiar with the example data. I didn’t include all the columns for this example. I just added a few of the relate the foreign keys and. Bit about the order itself. One thing to note when you create an in memory data source. It’s really nice because you can you can take advantage of all of the. All of the data bound objects in Servoy and all of the. Features you have surrounding the entity. So for example I. Can create calculations you see I made some here. This is how I was able to update. You saw the calculations updating as I was modifying the in memory. Portion of this entity methods. So any events so I handled the on record update. I’m going to show that in a little bit. It’s etc so you really get the full power of any data source here. Only this one we’re just populating dynamically. So you set up everything at at design time. And even this form itself if we look at the data source. The option for. So the in memory selection is here and then I they I prefix every in the W.S. So I picked W.S. orders and the same is true for. Any of the data providers you can see that. Relations work as well so I created relations from one memory data source to the other. So you can really. You can really do a complex setup with data that’s coming from sort of an arbitrary back end. So I made connections to customers and employees and even though the order details here which is also shown through. This relation. Okay let me let me jump back to where I was in the code here. So we’re at the point where. Where we just we just got our response object back from the web service. We got this array of 50 orders objects and and the details contain their in. So what I’m doing here is there’s there’s two ways to really go about do this. One is that you actually just work with a found set of the memory data source. The same way you would work with the found set of any other data source. And that’s what I’ve done here just for the sake of clarity. Another approach would be to populate a data set and then load the. Load the. The temp table in memory data source with the contents that data set the performance of that would actually be a better than going through the found set API in the overhead of. Of inserting records through the found set if I did just create that J as data set and then call create data source from that. It would be a bit faster but this is a bit clear to just to demonstrate. So one thing I’m doing is I’m deleting everything because I’m basically going to reload it. If I was in a pagination scenario. I might be paging in additional data so I might not delete all records. This may be something where I I step to the next chunk and I do that. I did want to do a pagination example here the back end supports it but I didn’t set it up here in the front end yet. I’m not going to do it. Anyway, so we iterate over the. The results of that object and you can actually see that. Response I’ve done in my get method. It actually returns. You can see it right here. I actually document so that it returns, you know, an array of results plus information about the pagination and an error if there was one. So I’m iterating over the results and at this point we’re just going to convert it to. To a record so I’m going to. I made another generic method because I’m calling this for different entities where I’m going to load an object into a record. And so I go over the. I go over all of the properties of the object here after I make my new record and really I’m just. checking to see if the target column exists. I’m assuming the mapping like the field names are the same or probably. In reality, you’d have some mapping where your. Target name might be different from the incoming name. I don’t know. Anyway, so I just to see if that the column exists in the target table, the in memory data source. And if it does exist, then I also check that column type and I cast the value into the right data type. So I’m parsing strings into dates into integers into numbers here. That’s an important step because everything you get back from your web service might not be. Not all that might be formatted the way you really needed it to be so in this case I’m doing a bit of formatting. And then I’m just setting the data provider saving data, etc. So this is sort of plain Jane creating records in Servoy and and just converting that object into a record. And that’s how we just load the data. Once you do that you’re working on. And then you’re working on a set up that should be very familiar. And that sort of takes us to. To the next example. By the way, I did the same thing for. For the. The order details. If I get to this part. I loaded the related customer by ID. And then the details were actually nested in the object. So I did those. I don’t want to jump into. It’s very much the same. Maybe I’ll show one of these methods here. For example, when I load the related customer. I call another. Web service request to if if it didn’t already if the customer wasn’t already loaded. So if it didn’t already. To load that customer by ID. And again, I put it through that generic get method and it sees that. Okay, it’s the ID and it just. Depends that to the end. So. If we go back to our. Running example. The next thing I wanted to show you is well. If you want to see what we want to see. Find we read everything to memory now that user starts doing things. Right. What happens if if we want to save what they did then update it through a web service. This is a very common scenario. That we see. That we see when. Particularly when integrating with like a legacy back end system. Right. Where we want to. We need to access data not through database, but indirectly. We need to get this up and I’ve put beside here a. A mirror which is actually connecting to the real database. This tab is connected to the cached results of the web service. The third tab is connected to the actual database. Because everything’s running in servo. We can take advantage of data broadcasting. And and show when the service when the update goes through the service. The data broadcast to the actual. The data is actually. I’m going to come back to the table. Form here that’s bound to the table. You can see when the table is updated. So what I’m going to do is come here and change the ship country to. I don’t USA. And at this point. You can see that this is still Germany. The cached. Web service results says USA. And I’m going to click the update button. I’ve just sent the. And then I’m going to go back to the database as updated. Via data broadcast to to the value. So this shows us that the web service did commit the. The change to the database. So what does that look like under the hood. In this case I went through that. That same. Test web service that we set up in the prior webinar. And if I open up the console here because I do some logging. You can see. This is the end point. This time it’s not a get request, but it’s a put request. And you can see the payload that was put in the put request. I only took what was changed. So here the ship country has changed. We identified the order was 10249. And then we sent the payload which gets serialized into a string. In the. In the. The body of the request. And then we get back in the response. We get the full. record. So if we wanted to resync it up. In this case I just see that there was no error and we say it updated. Okay. So. So that’s that’s what it looks like. In sort of the. HTTP part. If we want to look at the code. Again I wrapped everything in this web service scope. I have a method here called. Update order. This one’s a bit easier to follow because there aren’t so many. Subroutine’s for generic stuff. And again we set up the URL. Be sure to encode it. Once we get the URL set up. We create the put request. So put is generally in web in rest is generally meant for update. Whereas post is meant for insert. And so I added the header here for the content type. That’s kind of important. Because we also went back. We’re sending JSON and we want back. Jason. So I create this object. I iterate over the changes. If you never knew how to find out what’s changed in a record. You could just say get change data. The first column in that little data set just gives you the field names. Then it’s the old value in the new value. I don’t care about the old value. So I’m just sending. The. The new value and the actual value it is in the record. Or the name and then you saw that in the payload. Then you want to convert that object to a string. This is going to be almost every put or post. You’re going to call Jason stringify at some point and pass in the payload. This is where I printed it out. And I set the body content on the put request. And. And away we go. We executed. We get a response. We handle the error, all that stuff. If there was an error, we put a print out to a print out. To the console and just return false. So. So that’s pretty much it. For. You know, for this demo we showed. Reading and updating through the web service. Let’s jump over. I’ve prepared a few slides just to recap what we’ve seen. And. Then we’ll take some questions. Then we’ll take some questions. Meanwhile, if you guys do have any questions, then please start posting them now. So, but we get them in time to answer them. Thanks. Okay. I’ll do this quickly because I have just a few slides. So. Real quick, the HTTP plugin is where a lot of this stuff starts from. That’s how you can consume any web service. So. First of all, it’s responsible for creating a stateful client. Session. And that’s important to note that it is. I mean, you can just do isolated requests with the new client every time, but you can keep a stateful session going. Which can do things like. Access to session cookies. You can configure timeouts and even. Client proxy settings. The methods that it supports are the important ones for web services. A get is for getting a record or a list of records opposed is for an insert. Deleted is delete and put us for updating. It also supports less common stuff like head, head, patch and trace, which are rarely used. Every request that you generate through the plugin. You can add headers too. So we saw the content type header. You can of course execute it. One thing I didn’t show that I want to point out is that you can do. asynchronous mode. So the examples I showed were blocking. If the web service took 10 seconds to respond, then the code running in the client is waiting for 10 seconds. But you don’t have to do that. You can do the async mode. If I jump over to serve, oil and just show you on the HTTP client. Any of these requests like get request, you can see there’s execute. Plain execute request. And then there’s execute async where you’re always going to have this last argument as a function. It gets called back and just passes the response into it. Same approach. It just separates when you get the response. So you can do other stuff. Particularly when you send a request and you don’t really care about the response, I would always send it through async because that’ll just fire away. And you’ll keep moving with no delay. It supports HTTP basic authentication. It also supports a less common but the integrated windows authentication respan. Send the workspace. Name and all that stuff. If you need the OAuth, it’s best to do that through. We have a module for that to do it through the just through the plugin. It will be a lot of work. So it could be done. But it’d be no fun. Every request generates a standard response object. You’ll always get the status code. You should always check the status code. Make sure that it’s 200 OK. If it’s not, you want to handle that. You get access to any headers that were sent back. And of course, you want to get that response body. The response body can be taxed usually for web services. It’s serialized Jason. But it could also be binary as well. So the other part is the mem data sources that we saw. Just the what is part if you haven’t used them at all. It really is a fully compatible SQL database tables. In fact, it’s running on the app servers running HSQLDB, which is a Java implementation of a total in memory database. So there is a real database. There’s really SQL going to it. The JDBC driver connects to it. All that stuff. Nothing’s different than any other backend database. What you do is you define the schema design time. And then you load the table. And I want to say that you load the table. You don’t just load like a found set or a little cache data set. In the code that you’re running. Once you load the table, it stays there. And you can you can do stuff to it. It fully supports all the servoid data objects. So forms and found sets also runtime APIs. I mean, you could you could load 10,000 records into a memory data source. And then you can you can do finds and query builders and stuff like that against it. Of course, you can build relations and value this. We saw relations and calculations going on in the example I showed. So I had relations from one memory data source to another. You can have relations between, you know, real data sources and memory data sources. It’s quite flexible. One thing I didn’t show on the update was where that originally triggered from. I actually did it on the on record update event. So this is this is called when the order form is saving. So on record update is a trigger. So I’m not even triggering through the eye anytime. We just try to save that back to the memory data source. It will also then. Before it saves it will it will call the update order method that I showed you and send it off to the service. So just real quickly some best practices. I would say that if you’re going to consume a rest API, I would always try to wrap it and kind of mirror it in a client API. That way it shields you later down the road or other developers from the complexities of the HTTP and the service. So for example, we made things like load product, load customer where you pass in. Product ID or customer ID. I think I said this a few times. Always handle the status codes. There’s constants for all the different status codes in there. So you can use those. Make sure you’re encoding your URL. There’s a built in JavaScript function. It’s just in code URI and it’ll take a URL and then you can use it. You can use it as a file. There’s a built in JavaScript function. It’s just in code URI and it’ll take a URL and do things like substitute the escape characters. You want to specify the content type for Jason if you’re doing Jason especially on when you’re sending something in the payload. Also you want to parse the response body to get your payload. And then on the flip side when you’re sending something out with put or post, you want to do the inverse of that, which is Jason dot string of five and that’ll convert some complex object into into a string. Some useful links. So the HTTP plugin documentation is available on our wiki. We’ll post the recording of this webinar and we’ll post the sample solution in the slides to the Servoy forum that form thread that we keep running. I’ll leave those up a real quick. I want to just announce that that we’re going to do a fun detour with this tech series. We’re going to take a little bit of break because we’re going to head down to prepare some fun stuff and then we come back at the end of August and we’re going to do, I think a week long. Sort of mini series of these tech webinars all about the eight dot two release. So we’re going to go several days in a row with some examples of what’s new in eight dot two and that’ll be organized around the eight dot two release. So we’re going to take a break for about a month and then we’re going to come back on August 28th and we’re going to resume it with with a group of topics here.