Servoy QAPaaS – Quality Assurance as a Service – Launch
Servoy QAPaaS – Quality Assurance as a Service – Launch
Hello everyone. Welcome to the 30 minutes with Servoy Developer Series. I’m Sean Devlin. We are extra lucky because we typically do this every two weeks, but we just did one a week ago and now we’re back again with another topic. This is because we have had something working in the Servoy Labs and it’s important to Servoy and important to our customers, which is why we’ve invited our CEO, Ron Vandenberg, to speak about it. Before I turn it over to Ron, I just want to remind everyone that number one, this webinar is recorded. So if you like it so much that you want to watch it twice, we will record it and post it to our website. Also, please, we encourage you to put feedback and questions in the questions panel in your Go to Meeting Console and we will leave some room at the end of the webinar to deal with those. Ron, are you with us? Yeah, I’m here. Thanks, Sean. Yeah. Good. Good. Can you see much read? I can’t. I see that. That’s great. Is that you smoking the cigar? We’ll have to do this webinar. Okay. So thanks. Yeah, welcome everybody. Like Sean’s chat, we’ve been working hard on new features, new things. We talked a little bit about it at several world and now we’re ready to show it more and more through the world and really launch this thing. We’ve got the next slide. Yeah, maybe to start off a little bit a question to most of the audience, we probably are familiar with questions like this. Thanks for the new feature, but really introduce new bugs. Don’t you guys do with recursion testing? Yeah, we upgraded to a new version of the application and suddenly things stop working. It’s a pretty common question, I guess. And although people try to test as good as they can, it’s really difficult to get that that quality in place. We see out their problems, which happened through various reasons. Introductory of new features, leading bugs, taking tests or running tests, manual tests, take a lot of time. And often tests are actually run by customers. So typically when your customers give a lot of feedback about bugs, they’re basically testing for you and should you probably try to avoid that. And that’s not really the way to do things. It gets really released to a bad experience. And why do we think that continuous delivery is really, really important. For those of you who know what continuous delivery is, of course, it’s delivering the software in a constant flow to your customers. Your customers want new features fast. Appreciate your sales guys are out there selling new great things. Either the standard application typically, but also typically there’s a lot of questions from customers who want new features and they want them fast. They don’t want to wait. Why can’t you add that simple stuff tomorrow? So for vendors who want to release their new features really fast. On the other end, new kinds of versions of components in the totals that come out really, really fast. New versions of browser, plugins, Java, that come out really fast. Security breaches, they really happen fast or day. When you find them, there’s a lot of pressure on releasing new stuff fast. Other hand, we see that the really cycles of ISVs are slow, maybe due to things like testing and that’s a lot of work. And even if you release to your customers, the upgrade cycles of your customers go slow before they really adapt your new versions that takes a while. Now, you’re not alone out there. If you think, hey, I would love to start implementing new ways of working, 80% or even higher of mid-sized ISVs to not do automated QA, don’t do automated quality assurance, unit testing or end-to-end testing. All these small percentage to continuous build, continuous build in building daily or weekly automated from your source. And only also a small percentage of ISVs are doing automated end-to-end testing. A typical release cycle of large-size ERP applications are 12 to 18 months still today. Now, we also see things in industry changing rapidly. Google releases stuff every day. Sometimes we don’t even see it. If you run Chrome, that’s an example we often use. We typically don’t even see that that was upgraded. And the question is, what does that mean for complex apps? Ron, I don’t mean to interrupt you, but I have a question and maybe it’s a bit unfair, but I’m looking at this and I’m thinking, where’s Servoy? What’s our release cycle? Because we’re not releasing that to the public every day. Are we? Well, actually, we do. If you look at our print cycles, those are every three weeks. If you look at our release cycle, which we go on nightly builds, we do release to the public. But of course, I have a different print, footprint in terms of quality assurance. We release those every day or even more often a day. We have to continue delivery pipeline going in and we do have some customers that run the test against those daily or nightly builds. And that’s maybe also a nice spinoff of what we’re going to show you today is that in this pipeline that we’re building, you can work together with us in doing automated tests. So that’s fair, but unfair question. But yeah, the official releases for I think we do now every six weeks, that took me. But we do a release to the public every day. And we have customers who tap into that. That’s public. Well, maybe for example, I hear from a customer that they had a Servoy bug, they filed a case. It’s fixed in the next dot release of Servoy. And then that release comes out. And then they don’t upgrade. They don’t have a testing server that they do newer releases of Servoy with. And so then they’re waiting to a good time for them to upgrade maybe a few more weeks. And then they upgrade and they find the bug is 90% fixed, but there were some other issues that they go and they reopen the case. And then they wait for the next cycle again. And then it repeats. Are you saying that there’s a way to sort of short, short circuit that? Yeah, totally. That’s actually one of the main reasons why we try to go into this continuous release cycle mode, not just for our customers, but also ourselves and actually together. There was a world where that bug fix could be tested immediately after we fix it without human intervention, without the cost of going through the full release cycle, full test cycle of our customers. And we would have found together, let’s say the issue, and we could fix it within that strange print. And that’s what we’re, the infrastructure we’re making in the show today makes it totally possible. So yeah, we really try to tighten everything together because we’re at the end of the day together in the stack, if you can mention that way. Cool. So what do we want to bring you? We want to of course bring you increased competitiveness. We want to keep you competitive. I really bring you really rapidly cycles at the low cost. And it’s always been our mission. It’s always been our mission to enable you and to make our customers to develop software in a really rapid way. So not to have to worry about all kinds of underlying technologies and to keep track of that and to keep and to maintain all that and to keep it all tied together and and this whole technology stack, which is not about the user experience and it’s not about the features which you build. It’s really actually enable our customers to really focus on the user experience in the features that they’re delivering to customers. And don’t have to worry about infrastructure and cloud and all that technology that comes with it. And so what we’re doing here today is really in line with that. So what is the QA pass? QA stands for quality assurance and a pass and for platform and service. And we want to enable our customers to really start being in control of the quality of what they deliver to their customers. And basically is a complete test and proven stack for you to implement continuous integration with that comes into continuous testing and eventually also continue to implement to a production environment. Now, if we then look back to what is needed, what are the almost building blocks I level to do that. Of course, there needs to be some kind of infrastructure. And by the end of the day, this is service, which do things. Showing infrastructure. It is a test stack, a stack of server, which does this test for you. Of course, you need to continue integration delivery tools, test scripts. You need to write those. I’ll tell you a little bit about that on how we think we can help our customers make those scripts in a way more productive way than most technologies out of there. Of course, we want some visualization dashboard to tie things together. That gives you control and visibility. And of course, we need to integrate all those technologies and not just integrating, but also make some which is maintainable for future. So if new versions of for instance, Jenkins allowed it to come out in the future, we need to make sure that that’s future proof in a stack that we deliver to customers is going to stay future proof. And that, and what we did basically is we heard that question a lot from our customers. So we worked together with a couple of our customers in an alpha stage and developed what we show here today together with them. And actually today is a launch to invite customers to work together with us in a beta program to enhance this and make it probably more complete because the set of customers we work together only show a certain set of features which are required. And now we go into beta and try to make that the set of customers a little bit bigger. So again, what about the building blocks? Let me zoom in a little bit. So of course, in the front-end you need something which is unified like that’s where we start with that. We put in a case management system because by the end of the day, if people don’t have their own case management system, if it has fields that there’s needs to be injected to a system where it ends up as a task for somebody to look at it. But of course, if somebody has their own case management system that’s fine, think of a storage repository version management where you really talk about what pieces of the full stack end up in what version. So this infrastructure is not only to talk about what version is there in your solution or a wire file, but it’s about every piece of the puzzle which goes into a version, for instance, a version of Tomcat or even Java. Of course, you need to build the engine, something that really does the build jobs. You need a stack to do the testing. You know that we have unit testing into our environment already. But we build something which makes it possible to do easier end-to-end tests. Some preliminary performance tests, of course, use a acceptance testing. You need a control environment where you can say what version goes to, goes where. So, for instance, if you want to attend our environment or accept the environment to have a certain version of your stack, you need to control that and eventually need to deploy. Although you don’t need to buy the eventual deployment, but we have a lot of stuff there in place which we will end eventually also release in the topic. Hey, Ron, sorry, I’m just going to end up doing it again. But I see some of our customers with really talented technology people using Jenkins and even sometimes Selenium for the UI testing, Jenkins for the build management. I mean, these are all open source tools that are freely available and I think we deal with a lot of talented people. Why isn’t this something that our customers could just build and maintain on their own? You don’t think they can do it or what’s the… No, I think they can do it because by the end of the day, we also use a lot of open source technologies. But almost the same as we use a lot of open source technologies in, server yourself. It’s not a secret that we use hibernate for data buying to the database. But the reason for us to bring this all together and to deliver it as a very free service out of cloud because we think that it’s a lot of technology and we see customers struggling with it, not just struggling with it technically because technically people can learn it. We have, like you said, a lot of talented people at our customers, but by which you invest time and resources into building all this stuff, which you’re like, let’s say, next door neighbor in Servoy also needs. And we see a lot of customers, mid-size IVs, not having the time or resources to put the effort into that to build something like this. On the other hand, the need for it is growing and growing. So that’s why together with a couple of customers, we stepped into this to bring that all together. I think the feature that we have is pretty feature-rich already. And one thing with French with I don’t see a lot today with customers is of course customers can install Jenkins and build a wire file. And that’s pretty easy, but to make a unit testing and a unit testing or so the advent testing and do the antennas is testing in every OS and every browser combination. Or if you look at the resource management, how you’re going to put the board is using Kubernetics and Docker and make a scale and make it failover and make it zero down to upgrades. You keep on adding dimensions and dimensions and not even talking about maintaining it and making a a future proof so that if a new version of Docker comes out or Kubernetics, how are you going to make sure that also trickles down into your pipeline and the tools you use. So that’s why we stepped into this and work together with customers. Okay. What we see typical advanced IVs still happening a lot today. And I know a lot of customers yeah, we don’t do this, but we still see it a lot. Of course, most IVs have some kind of case issue management. A lot of them use GIRAN, some of you, some of them use ROGRA. I don’t see a lot more in a lot of companies who just work from their workspace, most of them use a storage repository, but a lot of them still generate a wire file from the ID or just a solution file and deploy it. There’s a lot of building blocks missing, which are eventually needed to pump up the quality and in production deliver things like scaling and availability and something which I rarely see things like solid log management and monitoring for instance. So a couple of mock-ups are in screenshots of things that are available. It’s of course in dashboard where you log in where you show an OCE, it’s an overview of things which are happening in the in the pipeline. Look at source management. We offer GIT and SGN in the pipeline so people can choice choose. And what I’m seeing is that we are trying to bring everything together in a unified front end. We put in case management where you manage your stories and features and bugs and tasks and everyone all that. For the case management, is that built-in or we have a lot of customers already using GIRA or fog bugs or whatever? It’s built in because for those customers who don’t have anything we need something where certain things like tasks will lend, but if people have their own, you can bring your own. We can integrate. Things like version management like I said, what version contains what needs to be in, monitoring about building and testing. So here you see the typical builds and what results are for overbuild, code quality measurement, so to measure what quality of the JavaScript is in your code base, things like we used things like KJ’s, Lintar stuff like that. So a lot more management which needs to really deep management into logging of not only serve or but Java and Tomcat and all the building blocks which are in the total stack. And the same goes from monitoring deep monitoring here. We only show usage of CPU a memory in a certain container, but using J makes for instance we can loop deeper into the server engine or can loop DPN to remember use of Tomcat or Java. And monitoring is already available in the QA pass so we can for instance in a in a Captain environment seed and then we’ll eventually of course be also available in production. This is the same thing. So if you look at what this QA pass infrastructure looks like, I talked a little bit about to make this future proof. The QA pass itself so our platform of course also comes through a build test except deployment environment and that is to make them to keep it future proof. So of course our customers are in the QA pass in our deploy the environment so live production but we have a build test and a acceptance environment running for the same. So Ron what in which when there’s a new Jenkins that has a build that has a security patch or something like that, then we would run a first of course from source we will configure a new version of Jenkins and then we will run it through our build test and acceptance environment in our acceptance environment we can see whether that would break anything. So our test will run there. So you QA the QA kind of thing. Yeah it’s like a compiler compiler or almost. Yes. Right. Right. And we have been pretty successful already in doing this. For all our consultants run their own pipeline for their projects and so we are being able to take out things there and continue to upgrade our pipeline. So our consultants of course run a production environment and in the meantime guys we’re building this QA pass our upgrading stuff to new versions and sometimes of course also add features. So if we deploy this to our customers so in this case a tenant is an iZ then those run in 100 percent separated silos. They don’t touch each other. The only thing which is controlling the total let’s say environments is of course our cloud environment which provisions basically vehicle name spaces for each iZ. So they can touch each other and they’re in completely separate environments. Today our staff runs on Amazon AWS but we potentially switch in a future to something else if needed. There’s a lot of staff talked about infrastructure which we think we can bring to our customers really easy and easy to step into. A big other of say almost hurdle is you can have this put without test scripts which test the functionality it’s useless. So the test scripts still need to be written and there are actually two type test scripts available. The unit SGA unit which is already built into the Servoy and some of you use. The I would say downside of that is that your application logic needs to be highly separated you really need units. So there’s sometimes a hurdle for customers who start using that. For the test for end-to-end test to start doing that typically requires a lot of low-level technical scripts. People use linear and protect their and all kinds of technologies and not only others at the low level but they are typically also fairly fragile. So what we added is a server-reparable rapid application development behavior driven development test. It’s a little bit into people know what the unit S is I presume which has been in share of way for eight years I think. I said not a lot of customer uses but because they need a high separation but what we added, so I’m not going to detail what it takes in terms of scripting to write low-level Selenium or protect their scripts but we added is a rapid application development and to end test script. What that basically does is this drives really high level the user flow through an application and so in this case this is really picked from a live running end-to-end test which we have to describe that you need to go to a certain URL in this case our demo application and you click on the navigation tab and then you click on a filtering tab and then you click on the date pick or you select a certain date you go to another list field and click an end date and also select the date. So this is the high level language. There’s no real technology involved you need no little bit about what the name of components is so here it says like for instance gallery main.naph which is the name of a component. I can show you a short movie of what that looks like. It’s running yeah it’s running. So basically we run here this script which I just showed and what that does and of course we show here now in a real UI mode it brings up our demo gallery and it really runs through all these tests. It basically clicking and scrolling and doing user interactions through an application and asserting at certain points where the certain values are what they should be. As you see it’s also possible to do that with pretty complex controls like a calendar picker or table views and it makes a way easy to write these these end to end test scripts. With end to end I don’t think I’ve been that these basically two things end to end for us means end to end the full stack. So you test from database to the user interaction repository you also test end to end from a user scenario. So somebody logs in, navigates who are sort of form and that’s a certain field and checks whether the values in the field is correct. So I think the test succeeded so you see everything green and of course this is showing the code running but in the throughout past you see a lot more high level reporting on this. So this is a view of a screen which shows the end result in this case it’s the test in the calendar component and you see that everything passed and of course when things fail you can even show screenshots or even videos or where it failed. For sure the people who get the task assigned to fix it they can figure out hey it went wrong and this in this screen this was the context and I need to look at it. So what’s that unique about? It’s really rapid application development. It has almost all of the underlying technologies. Generalize that. What we can do is we run, we can run this in a grid typically with tests during the running of a test it requires a lot of horsepower almost so we can run it in a grid. So because what you mimic is a browser really interacting with the server on which the system on the test runs we can do a bit multiple operations and browsers and also companies of that. We do performance testing during measurements so you can compare what it took to browse from run screen to another to run a certain feature in your application and compare that between tests and to see if that things start failing for instance or things become a lot slower. What’s really nice also and that’s not only about intent test but about the whole stack we can test against multiple versions of the story and we can even test against server nightly builds. So that’s what does it mean eventual we go to deployment and there’s a story because we do run this internally but we don’t give this to customers yet we want more experience with it and I think it will take some time to understand what it takes to get our customers in production on this I think eventually it would be nice to be able to help customers also not invest and having to invest time and resources into into setting this up. It’s about things that fail over how the scaling and zero data upgrades with the last I mean that you can upgrade to a new version of your solution or anything in the underlying stack without people having to log out and being automatically new sessions point to it to the new version and when everybody’s out to the old version and people are limited that’s that those are automatically put down. What it was working is deployment not just on cloud but also on premise we still have a lot of customers who have the requirement to deploy this stuff on premise for instance in healthcare or government where cloud is still not fully accepted and we have already architected and are working with one customer on to see how we can do that. I think in short for those who of you who think hey this is really the value is off you can start today you don’t have to wait we will not start any upfront cost there is just a page you go model and like I said the hurdle to start writing test scripts we have lowered to the lowest possible threshold this writing test scripts is really simple we have people in place who can train and help you on this or even maybe write the first 10 or 20 test scripts and then you can start running them immediately continuously you can choose code or even again future versions or so forth. Like I said bring a finding customers to join our beta we’re live with a couple of customers in alpha and we would like to make that set up people bigger to get more experience together and not just experience but to really understand what the features are that they require and that really adds value. So maybe if there’s time left Sean yeah we have some questions okay thanks good to you. I have to why I just interrupt you with my questions I don’t wait to the end. No I have to admit when I saw your slide deck I thought there’s no way this is going to be 30 minutes but you pulled it off so great job I’m too fast the Docker sorry. So we do have a few questions I think some of them you have answered along the way but I’ll restate them anyway. Can I test my source on a specific version of Servoyand we saw in one of the last slides the answer is that yes you can pick a version and you can even do the nightly snapshot. So that’s that’s I do for if you’re you know running in an older version but you still need to you’re still upgrading you know behind a bit then you can still test on a specific version. I think you answered this next question but what happens when a test fails? When a test fails we will inject it into the case system to as a task for somebody in a team to pick up and of course you need to figure out whether that’s false or negative positive so if that’s something which you grow in your solution or it broke somewhere due to what we did. Today our customers are responsible to investigate it themselves it’s also in the way we deploys you for today and maybe we have to look in the future where we can provide some kind of search where you can do that together I don’t know yet but we have to that’s one of the things which we which I would like to find out together with customers. And that’s that’s a I think a normal thing we expected because we want to tighten up the whole stack together because together so sky of oil and the solution we deliver something that the whole stack is the quality and it’s just one part of it and that’s why I think this is for everybody really important. One other question was about the test scripts themselves it is where the test scripts go can they also be versioned I guess that makes sense. The yes they go in the in the source repository and they are versioned and we’ve also set up a way to set up the tests correctly so typically of course with complex applications they rely heavily on the database. Maybe you also have tools in place and now best practices on how to set the database in the correct state before the test. So the setup and tear down part. Yep okay and then this last we’ve one more question here and I think you answered this already on the second the last slide. Is this something Servoy is hosting or we download it is it for hosting production as well? So we are hosting this and I think I mentioned it a little bit in the slides. So we host everything up until the access environment we’re working on production. We run it in a production ourselves for suffering for our demo sites. But I think today it’s not hard enough both technical but also legally. So for us to take on the data of your customers on our cloud that’s not an easy thing to overcome and we need to figure it out also together with customers.