Visual Regression Testing with BackstopJS
Visual regression testing with backstop. Je us my name is David Needham. I'm happy to be here to share a little bit about what I know about visual regression testing before I get started though I'd like to just start by saying that. This is not meant to be an exhaustive representation of everything that you can do with visual regression. I don't even claim to be an expert of backstop j/s I simply have a passion for sharing things that I learned. That helped me get started with tools at the very beginning so kind of getting your feet wet showing that it's not that difficult to try to run on your own so that you can play with it and learn and grow from there so that's the perspective that we're coming from today so my name is David Needham. I'm a developer advocate at Pantheon. I like to say that I sit between the developers that build Pantheon and the developers out in the community that use Pantheon so I take everything that I learned at Pantheon. Teach it to the community like everything that I learned by being in the community back to Pantheon so that we're making good decisions about features and functionality and all of that fun stuff as a result of me being in this position of living between developers in different locations living between developers and agencies and developers doing all three different technologies. I end up working with a pretty wide array of a varied number of libraries and techniques and strategies and so I'm happy to share a little bit of what I've learned about visual regression testing and specifically backstop. Jas alright so then this topic this session that we're talking about today is actually based on a class that we've done we've actually done a workshop that is a full day or a half day for a one and a half hour long session and I have kept the slides more or less as they are so we're going to be going through some of these slides pretty quickly but I wanted to keep them this way because we have all of our exercises in these slides and if you go get the slides you'll be able to go through all of these exercises on your own time as well so it's visual - regression dot David Needham me.
We'll get you directly to these slides so you can interact with them directly. I don't think you'll be able to follow along with the exercise isn't necessarily but if you pay attention to the slides or pay attention in the presentation I'll go through them pretty quickly and then you'll be able to do it on your own time in. Iram also thanks to Kevin. Thal this is being recorded so this video will show up on my website. David Needham got me so you'll be able to watch the video as many times as you'd like after today alright. So there's a bunch of stuff that you need to get started mostly node and NPM. You might already have that if you're already doing other things with node you need to install backstop in this case we're installing it globally so we can run backstop from whatever project that we're using it on. You can also install backstop unjustly particular project if you don't want to do it. Globally either way is fine and we will also be demonstrating a standalone node. App that one of my colleagues wrote so that if you have a QA team that maybe doesn't want to install backstop j/s themselves or know how kind of the ins and outs of how it works themselves. They'll be able to run a single command to execute a visual regression test which will help speed things up for them. A little bit make it lower barrier to entry for anyone to get started with. We'll be taking a look at that app later on as well and you'll have links to be able to download and modify that also. I do have a live version of a website and a development version of the website. So up here in my browser you can see. I have Pantheon demo comm which is a Pantheon theater website that we use for a lot of demos at Pantheon and I also have Pantheon - weekly demo site lando sites. I'm using lando for a local development environment they should be identical and another few minutes.
We'll be able to use visual regression testing to see if they're actually ident all right so with all the set up out of the way. Let's keep moving. Let's learn a little bit first about what visual regression testing. Actually is maybe you've heard the phrase before maybe you've heard of automated testing. Maybe you attended my colleague. Steve's session just right before this talking about automated testing well today. Let's start just by learning the basics of what visual regression specifically is and why you might be interested in them so first if we define regression you're thinking about a return to a former or less developed state so you are you built a project you launched it. You're very happy with it jump ahead something broke you make some sort of a change and now something isn't working anymore that's a regression you've went back from this completed state to something that's less than complete in what cases have you been like with visual regression. Well I can share some of my own stories. I have certainly had visual regression happen when I'm doing security updates you know in the Drupal community you have updates that come out about every Wednesday or so most of the time if you're doing a security update for your site you don't expect anything visually to change your patching some sort of a security bug everything should look more or less the same before and after that update and sometimes if it's been a while since you've done updates on your site something might break well that might cause a visual regression. Something might visually look different. Also when it comes to theming and sight building over time particularly with theming. We'll take a look at an example later. Like how many times have you made a little change and it affected more than you expected using visual regression testing. We'll be able to test for those things all right so I had some screenshots here and some some gifts that kind of showcase how this might be represented with doing routine updates and security updates.
This is a wordpress website. Don't hold it against me. We work with both. Drupal and WordPress on a regular basis and it's perfect example for both use cases so in this case we do have updates available. We click to apply the updates. So if you go to our live site kind of refresh your memory. What the live site looks like we kind of scroll through it you'll remind ourselves. I'm okay this is how it looks we then go over to our dev environment whether it's local or in a staging site we scroll through it we look at it. Did anyone notice the thing that changed on the page. No it is incredibly difficult for humans to catch very minor visual differences especially if you are testing like not just one page to the other but what if you have to test multiple pages. What if you have to test multiple viewports you know mobile versus desktop versus whatever so even if we set up a script to set them up side by side so they both scroll automatically. I know what I'm looking for so I caught it this time. But it's it's very subtle right. It's very difficult to this. If you're flipping through to catch what the difference might be and then yes if you're using a mobile is important. You're going to want to check for mobile as well to make sure that it's not breaking something. Visually well using visual regression testing. We have the ability to use machines computers. I don't know if I'd say like a I know Sara Lee but you can use a computer to compare two screenshots and highlight the things that are different between them so many times you're comparing a live version of a website versus a development version of a website. The machine will compare the two and then you see we have that sort of colored variation showing what has changed between them so we're using machine vision it's much much better than human vision and catching differences like this. It's also pretty easy to create. You'll see as we get started with backstop chess.
It's not difficult at all to create a test. It's not difficult to define additional pages or chain break points or anything like that and the test is relatively quick. You'll see that as well when we get to the demo section in this case we are using. Beck's Geass but that's one of many options for visual regression testing. You can do a google search. And you'll find many other options you will also find third-party hosted solutions where you plug in the URLs and the service just does it for you and there's often some sort of fee associated with that over time but there's lots of options for using visual regression so if we had used for that example earlier with that that website with the WordPress website in the update. We're using a backstop j/s here we get that little scrubber we can see visually. What's changed between the two. We can see that the search button went from just having the magnifying glass. Also having the word search in there from just doing that update it's much easier to see when we have the scrubber but the test is able to return a pass/fail so we can quickly see if it worked or it didn't and then the highlight will kind of direct us to the point we should be looking at all right so I actually had before we move on. I actually had an example. I want to share. It didn't happen this time because it was just through a very very perfect example but last time. I gave this presentation it was actually a week ago at five o'clock down in Kansas City. The projectors in the room were terrible absolutely terrible these slides used to have a black background and the text was either white or yellow and basically it was impossible to read on those projectors. It just did not work at all so at the last minute after. I plugged in and I saw it and like this isn't gonna work. I did some quick CSS changes to swap it out a higher contrast with white background and black text and you know slightly changing the font and all stuff and it was like oh man like I'm comparing now like before and after if only there is a way that I could tell if I'm catching everything or if this is changing things unexpectedly another slides.
I'm like wait a minute. I'm literally about to talk about visual regression testing so I ran a visual regression test. I wish I had saved it so I can use that in a demo now but I was able to catch like oh and a couple code examples I missed swapping out the CSS in that particular section so I was able to catch that and it was perfect. So you know you don't have to use visual regression testing just for and building a website. You can use it for anything on a website whether it's your website or even other people's websites you want to see what changed from one version to the other. You can absolutely do that all right. So taking a look at some benefits you do get to compare a reference versus test so it's assuming you have these two different versions. Typically the reference is your production site your live domain. The test is going to be whatever environment it is that you're working on whether it's you know a staging server or your local environment. We do also add multiple viewports. Whatever is important to you the more tests you have the longer it takes to run but it's important then you should definitely test for it. It is also possible to scan your entire website like I just said the more tests that you add the longer it takes to run however if it is important to you that every single page on your site is tested there is a backstop crawl extension or a plugin that you can get. That will write the test file for you after spy during scanning your entire site so in all the pages if you are doing that. I would probably discourage you from doing that because it will take a lot longer for the test to run. But if you if it's important that you test all the variations on your ear pages that you have. I recommend taking a sort of a snapshot of the ideal versions of each of your content types if you have a blog post if you have a biography page if you have a product page you know kind of pick one maybe even two of those perfect examples and then test those instead of testing every single you know dozens or hundreds of pages at once.
It is very helpful for error reporting. You'll be able to catch errors if you're scanning between the two and it does provide you with some pretty good support in the from the command line when you run this. If there's some sort of an error detected and again it is very easy to write these tests. There are some limits to visual regression. It's not perfect. In fact there is a lot of noise so as you are going through you might recall the side by side screen shot. I showed you are the the gif I showed you before. When we're doing this right here you might have noticed. Even though nothing in the header area changed. There's a whole bunch of noise up here. I think something changed in that header image. Even though nothing really changed this happens a lot when you're working with slideshows for example you know. Sometimes the slideshow doesn't load it exact at the same time and so as a result it's little sometimes you know say that there's a difference even though there is no difference. We'll probably see that in the demo that all just a minute also. Sometimes things are just different in. You don't know why they don't look different to you. It says it's different. There are ways to mitigate that we'll be telling it to ignore certain sections of our test another thing to keep in mind is if things are actively changing on the live version of the website people are adding new content if you have views that are constantly updating you know refreshing with new things in the site if you have a Twitter widget that's constantly updating with new streams. Those are all going to cause the test to fail when you really don't care in those cases we can't ignore those sections so that it doesn't impact the test so there all are a lot of false negatives that you really don't care about dynamic content is a pain ultimately at the end of the day visual regression testing is still worth it because it saves you a lot of time over manually just using your eyes to look yourself all right.
Let's start talking now about actually using oxj s so backstops. Yes you can go and grab from github we're able to use various render engines. Chrome your puppeteer and included by default. But there are a few others you can choose if you prefer not to use. Chrome or anything like that. You can also simulate interactions meaning if you would like to cause a number of buttons to be clicked. You know if you want to get to the end of a web form to validate that the. Thank You page after the web form is is still rendering properly. You can absolutely do that and then set that up in advance. You can also provide cookies so that you can have a logged in experience if you want to test the logged in version of your site. So you can create a dynamic cookie that's shared and it's able to use that to get into the admin interface it does provide visual reports that you can view and click and you saw the little scrubber that you can go back and forth on but it also provides command line reports. So if you're including this as part of a automated test you can have it pass/fail progressed to the next step. Hastur progressed to the next part of the script or not as well as G reports the ways that we've used this at Pantheon as far as like an automation step involves using circle CI but really any continuous integration process could be set up to use backstops pretty easily and then I mentioned earlier. We're installing this as global so we can run this as part of any project that we have on our computer but you could also set it up on just a singular project or package it as part of a node app and kind of run it from there and ultimately we're only going to be using three commands. We'll see it in just a minute but we're going to use backstop in it to initialize backstop in a particular project.
It generates the configuration file and it puts the the folders in place where it needs to go. We'll be using backstop reference to create the reference files so they look kind of looking at the production site creating screenshots of those so we have a representation of what we're looking at and then backstop test which which looks at our development site creat screenshots for that it compares it to our existing reference files and we'll take a look at those in just a moment so I mentioned when we do back stuff in it we end up with a backstop configuration. All of it is defined in your backstop. JSON file if we take a look at this it tries to be really readable and after I was able to get into it and see and kind of tweak some of these variables I found it. Pretty easy to work with. We can see here. I've highlighted some of the sections that I'd like to talk about at the very top. We have a section for viewports then viewports you define the viewports that you want you add a label you add width and height and it uses those to create individual reports for each page at each of these viewports. So if you care about any of these if you want to change them you can put in whatever you want below that we define scenarios and although this scenario really needs are these three sections that I've highlighted so we define a label what is the URL. What is the page that we're actually testing. What is the actual URL and what is the reference URL again the reference is the live production site the version that we're looking at at the perfect version versus the URL which is our development site below that we're going to see all sorts of other options for example. If you don't want the tests to run until the page is ready or a particular element is loaded. You can do that. You can set a delay. If things aren't quite loading at the same rate you can also hide or remove things from the page completely and when it refers to selectors you could put any typical like class.
IV element anything like that in there and then there's a bunch of other stuff and we're not gonna be using the other stuff so much. A lot of this is described on the github page. Though so if you'd like to do more advanced things the back stuff yes it's pretty easy to dig in and extend it even more all right so let's actually get in and start playing with it so. I'm gonna be doing this again. I don't expect anyone to necessarily have this ready to go. So you don't have to follow along but if you're following along in the video in the future feel free to pause and play and all that stuff and follow along if you'd like so. I've already installed backstop JS globally so now. I'm going to live it navigate to my local project via the command line and I'm gonna run back stop in it so I've already done that. I have a folder here with my site in it. I'm gonna do backstop in it to initialize backstop in this project right now. I'm going to use my local code editor to open this up and just show but is in the backstop JSON file all right if I open up backstop that JSON it looks pretty similar to the backstop to JSON that I showed you a little bit ago there's some other stuff in here so there's the viewports at the top or redefine the ones that we care about seeing there's this on ready and on before. J us where we can define other things in JavaScript. That need to get set up before we're ready to get started. We also have the label in it by default fills in the backstop website. We have a section here for cookies again if we want to do something with like logged in traffic or some sort of a session. That's active we could do that. I mean the URL the reference URL and all those other things so if I want to get started the example here I'm just going to follow it to keep it consistent but we have a label. Let's go ahead and add a name for our site the label. I'm going to choose the Pantheon Theater homepage and then for the URL.
The URL is going to be the local or development URL that's my Lando URL. So I'm going to grab that for my Lando site. Which is my local environments a sudden yes gonna go back and I want to grab now my reference URL which is my production site so that's my Pantheon demo com put that in the reference URL section and then I know from having done this before back. Stop in it will kind of get some stuff set up for you in advance that we really don't care to demo at this moment for example this cookies information if we leave it just as it is we're gonna get an error message. It's going to keep going through the test but I'm just going to remove this line because it's not something we actually need for this example like I mentioned earlier the only things that you really need for this scenario are these three lines the label the URL in the reference URL. You could literally remove everything else in this section and it would still run just fine and then there's all this stuff appear on ready scripts on before script similar things we don't need because we're not using them for this example. I'm going to remove them and just to avoid any extra text in the test when it runs and with that I think we are ready to begin. So create a new reference screenshot by running back stop reference so it is not going to our live site taking us screenshots of both the the wolf of the two viewports that I had defined some people have asked at this point about them creating these reference screenshots you know what if you commit these references as part of the project you think it repository that way you share them with other people on your team and then they can refer to the same reference you know through all testing. It's the same thing that made absolutely work. I haven't used it like that you know in a real world scenario before about it. I don't see why not all right and then the next thing is step number five we're going to run the tests with backstop tests and it's actually very helpful in the command line here because it says hey you've generated your references.
Now run backstop test to do the different. Court backs off test. It's now going to my development fight it is creating screenshots at each of those two viewports it's now generating the report and popping up in my default browser and as you can see. Oh there is actually a something that's changed actually. This is a new refresh real quick so something has changed on the live site this time on the live site we have a bright red button here on our Pantheon demo calm and so our backstops report actually caught that. It's funny because this is a live demo. That change happened since. I tested it right before class. So that's okay these things happen we can keep moving but so I was expecting these to be identical yeah I was not expecting to see a visual change. Something changed on the live site. That's okay but also something unexpected. Did change in mobile in that smaller viewport. The screen shot are the the slide show of that method at the top. It says there's something different between the two and it looks like by doing the scrubber that it's just not fully loaded in either stage it might be like milliseconds apart from each other and so as a result it's not slightly transparent on one hardly visible on the projector but what. I'm gonna try actually is going back to my backstop and maybe just setting a delay. So let's do maybe 800. This is in milliseconds. Let's try running it again. So we're gonna regenerate our reference files because when we change our test we want to regenerate the original ones that we're testing against because the parameters have changed creating new reference files from lives. Okay now we're going to do the tests which is going to create new reference our new screenshots of our development site. Okay perfect so the the slideshow at the top. There's no visual differences between that it's fully loaded now so that's great the button on the side is still bad and it's totally fine full disclosure that's actually something we use for the Pantheon demo that we do every week we quickly demonstrate a change in the Pantheon dashboard just by changing the color of the button so that's fine if we wanted to kind of match it up I could probably just do a do a just a check out on the theme folder there probably be enough pink check out okay so now let's let me actually go back to my sites do a refresh.
I'm expecting it to be red now and it is not red. Okay which makes me think maybe this version okay well hey we can fix this anyway if we go back to our sites go into the theme go into the museum theme change the CSS. We could make this red. We're gonna make it match. We'll go back in this case. We just changed our local environment. We didn't change the parameters of the test or anything like that so we can actually just do a back stop test. We don't have to create new reference files because we don't expect anything change done live again. We didn't change any of the parameters of the test all right and it looks like it passed so at the top you can see two tests past zero failed. It thinks these two are identical now which is great cool all right. Let's continue with the example so killer use case number two the first one as you recall was doing security updates on your site use case number two theming and site building. How often has someone said. Oh yeah this will just be a quick fix. I'm just gonna you know SSH you know I'm just gonna you know make this quick CSS change and then hours later you discover that it was a bigger thing. Who's done that before. Everyone right pretty much. So that's why visual regression testing can also save the day and kind of save you a lot of time so for that example that we just saw you know if we're going through if we wanted to change the the specifications of the test we're able to more accurately verify that whatever it is that we're changing is only affecting this this change and not breaking other parts of our site along the way so for example.
Let's just make some tweaks. Let's change this scenario a little bit. What if we want to change the viewports or the scenarios with individual. URLs that we're testing against well. Let's let's give it a try. Maybe for this example our marketing manager or someone says oh man like yeah that bright red button not liking it. What if we made it match our color scheme limit what if it was a yellow like a Pantheon yellow or something like that like well. We can certainly try it. We'll go in and maybe we'll say we'll have the hex code for the Pantheon yellow at the moment so let's just try a yellow but it's probably really ugly yellow. Let's just try that we'll do a refresh oof yeah. I think that's worse we do grab the Pantheon yellow real quick from the logo. Okay jump back to that. It's a tool called rulers. It lets you pad bounding boxes everything things to your screen but also to grab colors real quick cool to refresh all right a little bit better. Maybe if we make the text color black refresh alright so I mean you know it could probably be improved even better but I think this is a lot better than that bright red color right. So what if we say well. Let's do a test. We're expecting to see a difference this time back. Stop test and this is a this is something we expect right. We're going to see a difference between the live site now and our local site but we want to make sure that's only the change that we're expecting. We want to make sure that it is not affecting another part of the page or if we expand our test we can test other pages as well to verify. It's not breaking things there so yes we can see. This was the reference. This was the test we can see the dip. It's that bright obnoxious fuchsia pink color. I don't see anything else that is that pink color so I think we're safe looks pretty good. Well what if we want to make sure that our maybe our login page is not broken as a result of this we want to verify that that's working well.
I know from experience. Our login page is pretty ugly already we do. Have this button here but we also have a button down there. Let's make sure that it's not negatively affected by this in an automated test so we go into our examples our backstop j/s we go down to the scenario section. Remember we we do just need these three bits right here so. I'm going to grab part of this and go down and create a new scenario like that. I'm gonna say the Pantheon theater your zoo login and we're gonna say slash user slash users so now. We have two scenarios the front page and our login page. Maybe we also have decided that you know we. We don't really care about mobile at this point and tablet is really a little narrow so maybe we want to remove one of these tests or one of these viewports and we just want a desktop and maybe we say our ideal. Desktop range is 1600 by 940. Let's go through something out there okay. So because the parameters of our test changed because we added viewports and scenarios we need to create new reference files so we'll do backstop reference so it should be doing two. URLs with one viewport so I'm expecting and it says two of two alright that completed so now we'll do backstop test all right so yes the this is the front page on desktop. Looks fine to me on the login page. It also looks fine. It looks poorly gamed login page but besides that the only bright fuchsia pink color is on that login box on the the call-to-action box so it did only change that piece. It didn't break the login screen as far as I can tell yes question. The question was have. I seen any references to tracking like how many differences it finds on the page no. I haven't I think I haven't I. I don't know if it's possible or not I would expect it might have trouble because it's sort of like painting an area and like especially if something like the that example with like an image at the top it's painting like bits in pieces so you might get like hundreds of differences or maybe just one if it's an entire area that's painted that color so that's just fix little differences.
It is just checking pixel differences between two screenshots. Yep all right well so let's keep going so this was an example where we changed viewports in the scenarios we regenerated our reference file because our test parameters changed and then we ran back. Stop test if you're not changing the parameters of the test. If you know you're not expecting anything to be changing on live you can just run back. Stop test over and over and over and over again and it's always going to be checking your local versus those images that it created up the reference earlier all right so we only have a few more minutes. But let's talk briefly about automating this. Let's use let's use something some continuous integration process. So you can certainly everything. I've done up to. This point is a manual process. You can just run on your computer even if the rest of your team isn't using backstopped. Yes you can verify that what you're doing is that introducing regression or use it for side projects or whatever you want when it comes to continuous integration you might include backstop. J s and visual regression testing to verify that something is not regressing before continuing to the next step so for example at Pantheon we have a we have a command line tool called terminus and we have a script example script available that will fully automate the process of doing security updates it checks for security updates as part of a continuous integration step if there are updates found it applies those updates into a branch. Pantheon we call them multi-day of environments it will then do visual regression testing performance testing accessibility testing and even B hat testing. And if every test passes with zero failures zero pixel differences between between that development environment and the live site.
It will actually create backups on our live sites and test site. Deploy it up to live so fully hands-off fully automated by testing that extensively. We're able to increase our confidence such that we feel comfortable automating that process. Of course it is just a script you can modify however you want but I think it's a great example of automating the process with continuous integration ultimately if you are currently doing QA manually and you're struggling through it it can be good to think about what things have your what parts of your job. You could use. Robots computers do things that they're better at faster at more efficient at that you could include as part of your workflow so that you don't have to do it manually anymore so this was actually the example. I got to a little bit early but you can definitely go grab this grip take a look at it. It is currently anticipating that using it's anticipating that using Pantheon you can take it modify it if I'm github however you want so it. Check for updates. On cron generates the new environment applies the updates make backup and then applies it notifies you that it's done the second example so we actually ran a workshop a full-day workshop at Drupal column called automated workflows and Drupal AIDS with github composer and circle. CI with this particular class. We went through demonstrated a workflow where we added a module with composer. I think of the address module because it's a great place to get started people use a lot so on a Drupal 8 site we added the address module composer. We pushed the branch that we had created for that up to github and create a pull request as part of that the build ran automatically with circle. CI it compiled the sites with composer pulled in all the dependencies kind of tested. It built it and placed this perfect version over. Pantheon pre-built tested and all that stuff ready to go and the visual regression test and all the other tests appeared in the pull requests automatically so all that the developer.
How'd you do use composer. Make the pull request. All the testing happened automatically and it shows up for their lead developer or their project manager. Whomever to verify that the things change that they're expecting the change if this is something that's interesting to you. I am expecting. We'll probably do this class sometime again. And we are right now talking about doing this as a either. A short webinar series like maybe three one-hour chunks or maybe a single three-hour training that we would do online. If you're interested maybe come up to me afterwards or shoot me an email or something. Let me know and we'll get you on the list for that. It's not something we charge for. So if like to learn more about this process let us know okay and then finally the standalone app so. I don't have time to go through this right now. Unfortunately but let me just talk about what it would look like so the scenario again is that you have a QA team. You have someone who doesn't feel comfortable running back stop gs themselves or configuring the mo file you can give them a anode app. That's pretty kind of set up pre-configured for your project so that every time that you run you know and node run start or whatever npm run start or whatever it might be that you run or as part of another operation you know maybe you put a button on their desktop and they click the button and it runs the test automatically and just pops up in their browser and they have a test that just ran because they they fired it off so all of these examples are here you can go download the examples. Run them on your own check it out see if you like it. The examples include the first one which is just it's just a site and you run it and it runs and it's done and then there's a multiple site example where you run it and ask you which of these three or four sites would you like to test you choose the one you want and then it goes and tests it and then you can run all in the next block but cool so any questions.
Yes hearts yeah so so. I was so in the mini sense that I've run this on this session and workshop that slideshow at the top gives me varying degrees of difficulty and I can't consistently figure out how to fix it every single time. I was fortunate this time that simply setting the delay was enough. I was able to say you know. Wait this long and then take a screenshot and then it doesn't catch it anymore occasionally. I'll have to remove it or hide it so for example. Let me just do that real quick and we'll see how that works so here back in my backstop. JSON file we have a section for yes. Thank you right here. Hide selectors so if we just put in a selector maybe in flex slider is the like service to be put in. Hi selectors like this and we go back. We're gonna have to do back. Stop reference to regenerate our reference files long rating for that looking at this so we have hide selectors which will just do a. It'll do a hide but kind of keep the element in place so nothing on the page should move. It'll just like blank out that area so it doesn't affect the test versus remove selectors which removes the code and everything moves to account for that thing missing from the page at that point so. I've found hide selectors to be a little better but sometimes you just need to remove something for other reasons hopeful that completed so backstop test. No so again with that scenario that random happenstance situation where my slides were different all that I did was go into this back step that JSON file to find my live slides versus my local slides. I did a test so I didn't have to set up a brand new project or do anything fancy. You don't have to use it for a website. You can use it for anything but as you can see we. We hid that slideshow element at the very top and oh I forgot to add the the hide element for the second scenario but for the first one. It's it's not there anymore.
Austin is these basically the same one you're really gonna see if the Sun of fame when you're running it on Thea VI so I it's pretty much the same thing. Yeah yeah it depends on exactly what you wanted to do. You're you're able to set up your backstop that JSON file so you can pass through like variables you know so I for example if you take a look at the code for the node app which you do if you follow the exercise you actually go in and you have a separate backstop file that is like sites to be tested and you define all the important elements for each site in a object or an array or something and then each of those is passed into a generic backstop that JSON file via with variables. And all those things so depending on what it is that you want to do and how complex you want to be you. Could you can do other things. But it's pretty similar to this the better performance next got some sight the see. I whipped the performance it for an extent to the fight with the eye. What do you mean if you brought a lot of pesticide what how are you. Haven't we like performance of the site or performance of the test. I mean you saw you know adding its it's cumulative or exponential but it like for each test it takes you know if each test takes five seconds or ten seconds. Each viewport multiplies that each scenario. You know so it. It grows pretty quickly. Yeah yeah and so in the interest of making it something you'll actually use and not be annoyed by you. Want to keep it pretty lean but if you have this as part of an automation step that you run every time that you do a pull request you don't want to forget anything and so it might run longer but you kind of forgive it because you just make the pull request and go get some coffee and it's not something you necessarily do every day all right well thank you all very much for attending and I hope you all the great rest of camera.
We'll get you directly to these slides so you can interact with them directly. I don't think you'll be able to follow along with the exercise isn't necessarily but if you pay attention to the slides or pay attention in the presentation I'll go through them pretty quickly and then you'll be able to do it on your own time in. Iram also thanks to Kevin. Thal this is being recorded so this video will show up on my website. David Needham got me so you'll be able to watch the video as many times as you'd like after today alright. So there's a bunch of stuff that you need to get started mostly node and NPM. You might already have that if you're already doing other things with node you need to install backstop in this case we're installing it globally so we can run backstop from whatever project that we're using it on. You can also install backstop unjustly particular project if you don't want to do it. Globally either way is fine and we will also be demonstrating a standalone node. App that one of my colleagues wrote so that if you have a QA team that maybe doesn't want to install backstop j/s themselves or know how kind of the ins and outs of how it works themselves. They'll be able to run a single command to execute a visual regression test which will help speed things up for them. A little bit make it lower barrier to entry for anyone to get started with. We'll be taking a look at that app later on as well and you'll have links to be able to download and modify that also. I do have a live version of a website and a development version of the website. So up here in my browser you can see. I have Pantheon demo comm which is a Pantheon theater website that we use for a lot of demos at Pantheon and I also have Pantheon - weekly demo site lando sites. I'm using lando for a local development environment they should be identical and another few minutes.
We'll be able to use visual regression testing to see if they're actually ident all right so with all the set up out of the way. Let's keep moving. Let's learn a little bit first about what visual regression testing. Actually is maybe you've heard the phrase before maybe you've heard of automated testing. Maybe you attended my colleague. Steve's session just right before this talking about automated testing well today. Let's start just by learning the basics of what visual regression specifically is and why you might be interested in them so first if we define regression you're thinking about a return to a former or less developed state so you are you built a project you launched it. You're very happy with it jump ahead something broke you make some sort of a change and now something isn't working anymore that's a regression you've went back from this completed state to something that's less than complete in what cases have you been like with visual regression. Well I can share some of my own stories. I have certainly had visual regression happen when I'm doing security updates you know in the Drupal community you have updates that come out about every Wednesday or so most of the time if you're doing a security update for your site you don't expect anything visually to change your patching some sort of a security bug everything should look more or less the same before and after that update and sometimes if it's been a while since you've done updates on your site something might break well that might cause a visual regression. Something might visually look different. Also when it comes to theming and sight building over time particularly with theming. We'll take a look at an example later. Like how many times have you made a little change and it affected more than you expected using visual regression testing. We'll be able to test for those things all right so I had some screenshots here and some some gifts that kind of showcase how this might be represented with doing routine updates and security updates.
This is a wordpress website. Don't hold it against me. We work with both. Drupal and WordPress on a regular basis and it's perfect example for both use cases so in this case we do have updates available. We click to apply the updates. So if you go to our live site kind of refresh your memory. What the live site looks like we kind of scroll through it you'll remind ourselves. I'm okay this is how it looks we then go over to our dev environment whether it's local or in a staging site we scroll through it we look at it. Did anyone notice the thing that changed on the page. No it is incredibly difficult for humans to catch very minor visual differences especially if you are testing like not just one page to the other but what if you have to test multiple pages. What if you have to test multiple viewports you know mobile versus desktop versus whatever so even if we set up a script to set them up side by side so they both scroll automatically. I know what I'm looking for so I caught it this time. But it's it's very subtle right. It's very difficult to this. If you're flipping through to catch what the difference might be and then yes if you're using a mobile is important. You're going to want to check for mobile as well to make sure that it's not breaking something. Visually well using visual regression testing. We have the ability to use machines computers. I don't know if I'd say like a I know Sara Lee but you can use a computer to compare two screenshots and highlight the things that are different between them so many times you're comparing a live version of a website versus a development version of a website. The machine will compare the two and then you see we have that sort of colored variation showing what has changed between them so we're using machine vision it's much much better than human vision and catching differences like this. It's also pretty easy to create. You'll see as we get started with backstop chess.
It's not difficult at all to create a test. It's not difficult to define additional pages or chain break points or anything like that and the test is relatively quick. You'll see that as well when we get to the demo section in this case we are using. Beck's Geass but that's one of many options for visual regression testing. You can do a google search. And you'll find many other options you will also find third-party hosted solutions where you plug in the URLs and the service just does it for you and there's often some sort of fee associated with that over time but there's lots of options for using visual regression so if we had used for that example earlier with that that website with the WordPress website in the update. We're using a backstop j/s here we get that little scrubber we can see visually. What's changed between the two. We can see that the search button went from just having the magnifying glass. Also having the word search in there from just doing that update it's much easier to see when we have the scrubber but the test is able to return a pass/fail so we can quickly see if it worked or it didn't and then the highlight will kind of direct us to the point we should be looking at all right so I actually had before we move on. I actually had an example. I want to share. It didn't happen this time because it was just through a very very perfect example but last time. I gave this presentation it was actually a week ago at five o'clock down in Kansas City. The projectors in the room were terrible absolutely terrible these slides used to have a black background and the text was either white or yellow and basically it was impossible to read on those projectors. It just did not work at all so at the last minute after. I plugged in and I saw it and like this isn't gonna work. I did some quick CSS changes to swap it out a higher contrast with white background and black text and you know slightly changing the font and all stuff and it was like oh man like I'm comparing now like before and after if only there is a way that I could tell if I'm catching everything or if this is changing things unexpectedly another slides.
I'm like wait a minute. I'm literally about to talk about visual regression testing so I ran a visual regression test. I wish I had saved it so I can use that in a demo now but I was able to catch like oh and a couple code examples I missed swapping out the CSS in that particular section so I was able to catch that and it was perfect. So you know you don't have to use visual regression testing just for and building a website. You can use it for anything on a website whether it's your website or even other people's websites you want to see what changed from one version to the other. You can absolutely do that all right. So taking a look at some benefits you do get to compare a reference versus test so it's assuming you have these two different versions. Typically the reference is your production site your live domain. The test is going to be whatever environment it is that you're working on whether it's you know a staging server or your local environment. We do also add multiple viewports. Whatever is important to you the more tests you have the longer it takes to run but it's important then you should definitely test for it. It is also possible to scan your entire website like I just said the more tests that you add the longer it takes to run however if it is important to you that every single page on your site is tested there is a backstop crawl extension or a plugin that you can get. That will write the test file for you after spy during scanning your entire site so in all the pages if you are doing that. I would probably discourage you from doing that because it will take a lot longer for the test to run. But if you if it's important that you test all the variations on your ear pages that you have. I recommend taking a sort of a snapshot of the ideal versions of each of your content types if you have a blog post if you have a biography page if you have a product page you know kind of pick one maybe even two of those perfect examples and then test those instead of testing every single you know dozens or hundreds of pages at once.
It is very helpful for error reporting. You'll be able to catch errors if you're scanning between the two and it does provide you with some pretty good support in the from the command line when you run this. If there's some sort of an error detected and again it is very easy to write these tests. There are some limits to visual regression. It's not perfect. In fact there is a lot of noise so as you are going through you might recall the side by side screen shot. I showed you are the the gif I showed you before. When we're doing this right here you might have noticed. Even though nothing in the header area changed. There's a whole bunch of noise up here. I think something changed in that header image. Even though nothing really changed this happens a lot when you're working with slideshows for example you know. Sometimes the slideshow doesn't load it exact at the same time and so as a result it's little sometimes you know say that there's a difference even though there is no difference. We'll probably see that in the demo that all just a minute also. Sometimes things are just different in. You don't know why they don't look different to you. It says it's different. There are ways to mitigate that we'll be telling it to ignore certain sections of our test another thing to keep in mind is if things are actively changing on the live version of the website people are adding new content if you have views that are constantly updating you know refreshing with new things in the site if you have a Twitter widget that's constantly updating with new streams. Those are all going to cause the test to fail when you really don't care in those cases we can't ignore those sections so that it doesn't impact the test so there all are a lot of false negatives that you really don't care about dynamic content is a pain ultimately at the end of the day visual regression testing is still worth it because it saves you a lot of time over manually just using your eyes to look yourself all right.
Let's start talking now about actually using oxj s so backstops. Yes you can go and grab from github we're able to use various render engines. Chrome your puppeteer and included by default. But there are a few others you can choose if you prefer not to use. Chrome or anything like that. You can also simulate interactions meaning if you would like to cause a number of buttons to be clicked. You know if you want to get to the end of a web form to validate that the. Thank You page after the web form is is still rendering properly. You can absolutely do that and then set that up in advance. You can also provide cookies so that you can have a logged in experience if you want to test the logged in version of your site. So you can create a dynamic cookie that's shared and it's able to use that to get into the admin interface it does provide visual reports that you can view and click and you saw the little scrubber that you can go back and forth on but it also provides command line reports. So if you're including this as part of a automated test you can have it pass/fail progressed to the next step. Hastur progressed to the next part of the script or not as well as G reports the ways that we've used this at Pantheon as far as like an automation step involves using circle CI but really any continuous integration process could be set up to use backstops pretty easily and then I mentioned earlier. We're installing this as global so we can run this as part of any project that we have on our computer but you could also set it up on just a singular project or package it as part of a node app and kind of run it from there and ultimately we're only going to be using three commands. We'll see it in just a minute but we're going to use backstop in it to initialize backstop in a particular project.
It generates the configuration file and it puts the the folders in place where it needs to go. We'll be using backstop reference to create the reference files so they look kind of looking at the production site creating screenshots of those so we have a representation of what we're looking at and then backstop test which which looks at our development site creat screenshots for that it compares it to our existing reference files and we'll take a look at those in just a moment so I mentioned when we do back stuff in it we end up with a backstop configuration. All of it is defined in your backstop. JSON file if we take a look at this it tries to be really readable and after I was able to get into it and see and kind of tweak some of these variables I found it. Pretty easy to work with. We can see here. I've highlighted some of the sections that I'd like to talk about at the very top. We have a section for viewports then viewports you define the viewports that you want you add a label you add width and height and it uses those to create individual reports for each page at each of these viewports. So if you care about any of these if you want to change them you can put in whatever you want below that we define scenarios and although this scenario really needs are these three sections that I've highlighted so we define a label what is the URL. What is the page that we're actually testing. What is the actual URL and what is the reference URL again the reference is the live production site the version that we're looking at at the perfect version versus the URL which is our development site below that we're going to see all sorts of other options for example. If you don't want the tests to run until the page is ready or a particular element is loaded. You can do that. You can set a delay. If things aren't quite loading at the same rate you can also hide or remove things from the page completely and when it refers to selectors you could put any typical like class.
IV element anything like that in there and then there's a bunch of other stuff and we're not gonna be using the other stuff so much. A lot of this is described on the github page. Though so if you'd like to do more advanced things the back stuff yes it's pretty easy to dig in and extend it even more all right so let's actually get in and start playing with it so. I'm gonna be doing this again. I don't expect anyone to necessarily have this ready to go. So you don't have to follow along but if you're following along in the video in the future feel free to pause and play and all that stuff and follow along if you'd like so. I've already installed backstop JS globally so now. I'm going to live it navigate to my local project via the command line and I'm gonna run back stop in it so I've already done that. I have a folder here with my site in it. I'm gonna do backstop in it to initialize backstop in this project right now. I'm going to use my local code editor to open this up and just show but is in the backstop JSON file all right if I open up backstop that JSON it looks pretty similar to the backstop to JSON that I showed you a little bit ago there's some other stuff in here so there's the viewports at the top or redefine the ones that we care about seeing there's this on ready and on before. J us where we can define other things in JavaScript. That need to get set up before we're ready to get started. We also have the label in it by default fills in the backstop website. We have a section here for cookies again if we want to do something with like logged in traffic or some sort of a session. That's active we could do that. I mean the URL the reference URL and all those other things so if I want to get started the example here I'm just going to follow it to keep it consistent but we have a label. Let's go ahead and add a name for our site the label. I'm going to choose the Pantheon Theater homepage and then for the URL.
The URL is going to be the local or development URL that's my Lando URL. So I'm going to grab that for my Lando site. Which is my local environments a sudden yes gonna go back and I want to grab now my reference URL which is my production site so that's my Pantheon demo com put that in the reference URL section and then I know from having done this before back. Stop in it will kind of get some stuff set up for you in advance that we really don't care to demo at this moment for example this cookies information if we leave it just as it is we're gonna get an error message. It's going to keep going through the test but I'm just going to remove this line because it's not something we actually need for this example like I mentioned earlier the only things that you really need for this scenario are these three lines the label the URL in the reference URL. You could literally remove everything else in this section and it would still run just fine and then there's all this stuff appear on ready scripts on before script similar things we don't need because we're not using them for this example. I'm going to remove them and just to avoid any extra text in the test when it runs and with that I think we are ready to begin. So create a new reference screenshot by running back stop reference so it is not going to our live site taking us screenshots of both the the wolf of the two viewports that I had defined some people have asked at this point about them creating these reference screenshots you know what if you commit these references as part of the project you think it repository that way you share them with other people on your team and then they can refer to the same reference you know through all testing. It's the same thing that made absolutely work. I haven't used it like that you know in a real world scenario before about it. I don't see why not all right and then the next thing is step number five we're going to run the tests with backstop tests and it's actually very helpful in the command line here because it says hey you've generated your references.
Now run backstop test to do the different. Court backs off test. It's now going to my development fight it is creating screenshots at each of those two viewports it's now generating the report and popping up in my default browser and as you can see. Oh there is actually a something that's changed actually. This is a new refresh real quick so something has changed on the live site this time on the live site we have a bright red button here on our Pantheon demo calm and so our backstops report actually caught that. It's funny because this is a live demo. That change happened since. I tested it right before class. So that's okay these things happen we can keep moving but so I was expecting these to be identical yeah I was not expecting to see a visual change. Something changed on the live site. That's okay but also something unexpected. Did change in mobile in that smaller viewport. The screen shot are the the slide show of that method at the top. It says there's something different between the two and it looks like by doing the scrubber that it's just not fully loaded in either stage it might be like milliseconds apart from each other and so as a result it's not slightly transparent on one hardly visible on the projector but what. I'm gonna try actually is going back to my backstop and maybe just setting a delay. So let's do maybe 800. This is in milliseconds. Let's try running it again. So we're gonna regenerate our reference files because when we change our test we want to regenerate the original ones that we're testing against because the parameters have changed creating new reference files from lives. Okay now we're going to do the tests which is going to create new reference our new screenshots of our development site. Okay perfect so the the slideshow at the top. There's no visual differences between that it's fully loaded now so that's great the button on the side is still bad and it's totally fine full disclosure that's actually something we use for the Pantheon demo that we do every week we quickly demonstrate a change in the Pantheon dashboard just by changing the color of the button so that's fine if we wanted to kind of match it up I could probably just do a do a just a check out on the theme folder there probably be enough pink check out okay so now let's let me actually go back to my sites do a refresh.
I'm expecting it to be red now and it is not red. Okay which makes me think maybe this version okay well hey we can fix this anyway if we go back to our sites go into the theme go into the museum theme change the CSS. We could make this red. We're gonna make it match. We'll go back in this case. We just changed our local environment. We didn't change the parameters of the test or anything like that so we can actually just do a back stop test. We don't have to create new reference files because we don't expect anything change done live again. We didn't change any of the parameters of the test all right and it looks like it passed so at the top you can see two tests past zero failed. It thinks these two are identical now which is great cool all right. Let's continue with the example so killer use case number two the first one as you recall was doing security updates on your site use case number two theming and site building. How often has someone said. Oh yeah this will just be a quick fix. I'm just gonna you know SSH you know I'm just gonna you know make this quick CSS change and then hours later you discover that it was a bigger thing. Who's done that before. Everyone right pretty much. So that's why visual regression testing can also save the day and kind of save you a lot of time so for that example that we just saw you know if we're going through if we wanted to change the the specifications of the test we're able to more accurately verify that whatever it is that we're changing is only affecting this this change and not breaking other parts of our site along the way so for example.
Let's just make some tweaks. Let's change this scenario a little bit. What if we want to change the viewports or the scenarios with individual. URLs that we're testing against well. Let's let's give it a try. Maybe for this example our marketing manager or someone says oh man like yeah that bright red button not liking it. What if we made it match our color scheme limit what if it was a yellow like a Pantheon yellow or something like that like well. We can certainly try it. We'll go in and maybe we'll say we'll have the hex code for the Pantheon yellow at the moment so let's just try a yellow but it's probably really ugly yellow. Let's just try that we'll do a refresh oof yeah. I think that's worse we do grab the Pantheon yellow real quick from the logo. Okay jump back to that. It's a tool called rulers. It lets you pad bounding boxes everything things to your screen but also to grab colors real quick cool to refresh all right a little bit better. Maybe if we make the text color black refresh alright so I mean you know it could probably be improved even better but I think this is a lot better than that bright red color right. So what if we say well. Let's do a test. We're expecting to see a difference this time back. Stop test and this is a this is something we expect right. We're going to see a difference between the live site now and our local site but we want to make sure that's only the change that we're expecting. We want to make sure that it is not affecting another part of the page or if we expand our test we can test other pages as well to verify. It's not breaking things there so yes we can see. This was the reference. This was the test we can see the dip. It's that bright obnoxious fuchsia pink color. I don't see anything else that is that pink color so I think we're safe looks pretty good. Well what if we want to make sure that our maybe our login page is not broken as a result of this we want to verify that that's working well.
I know from experience. Our login page is pretty ugly already we do. Have this button here but we also have a button down there. Let's make sure that it's not negatively affected by this in an automated test so we go into our examples our backstop j/s we go down to the scenario section. Remember we we do just need these three bits right here so. I'm going to grab part of this and go down and create a new scenario like that. I'm gonna say the Pantheon theater your zoo login and we're gonna say slash user slash users so now. We have two scenarios the front page and our login page. Maybe we also have decided that you know we. We don't really care about mobile at this point and tablet is really a little narrow so maybe we want to remove one of these tests or one of these viewports and we just want a desktop and maybe we say our ideal. Desktop range is 1600 by 940. Let's go through something out there okay. So because the parameters of our test changed because we added viewports and scenarios we need to create new reference files so we'll do backstop reference so it should be doing two. URLs with one viewport so I'm expecting and it says two of two alright that completed so now we'll do backstop test all right so yes the this is the front page on desktop. Looks fine to me on the login page. It also looks fine. It looks poorly gamed login page but besides that the only bright fuchsia pink color is on that login box on the the call-to-action box so it did only change that piece. It didn't break the login screen as far as I can tell yes question. The question was have. I seen any references to tracking like how many differences it finds on the page no. I haven't I think I haven't I. I don't know if it's possible or not I would expect it might have trouble because it's sort of like painting an area and like especially if something like the that example with like an image at the top it's painting like bits in pieces so you might get like hundreds of differences or maybe just one if it's an entire area that's painted that color so that's just fix little differences.
It is just checking pixel differences between two screenshots. Yep all right well so let's keep going so this was an example where we changed viewports in the scenarios we regenerated our reference file because our test parameters changed and then we ran back. Stop test if you're not changing the parameters of the test. If you know you're not expecting anything to be changing on live you can just run back. Stop test over and over and over and over again and it's always going to be checking your local versus those images that it created up the reference earlier all right so we only have a few more minutes. But let's talk briefly about automating this. Let's use let's use something some continuous integration process. So you can certainly everything. I've done up to. This point is a manual process. You can just run on your computer even if the rest of your team isn't using backstopped. Yes you can verify that what you're doing is that introducing regression or use it for side projects or whatever you want when it comes to continuous integration you might include backstop. J s and visual regression testing to verify that something is not regressing before continuing to the next step so for example at Pantheon we have a we have a command line tool called terminus and we have a script example script available that will fully automate the process of doing security updates it checks for security updates as part of a continuous integration step if there are updates found it applies those updates into a branch. Pantheon we call them multi-day of environments it will then do visual regression testing performance testing accessibility testing and even B hat testing. And if every test passes with zero failures zero pixel differences between between that development environment and the live site.
It will actually create backups on our live sites and test site. Deploy it up to live so fully hands-off fully automated by testing that extensively. We're able to increase our confidence such that we feel comfortable automating that process. Of course it is just a script you can modify however you want but I think it's a great example of automating the process with continuous integration ultimately if you are currently doing QA manually and you're struggling through it it can be good to think about what things have your what parts of your job. You could use. Robots computers do things that they're better at faster at more efficient at that you could include as part of your workflow so that you don't have to do it manually anymore so this was actually the example. I got to a little bit early but you can definitely go grab this grip take a look at it. It is currently anticipating that using it's anticipating that using Pantheon you can take it modify it if I'm github however you want so it. Check for updates. On cron generates the new environment applies the updates make backup and then applies it notifies you that it's done the second example so we actually ran a workshop a full-day workshop at Drupal column called automated workflows and Drupal AIDS with github composer and circle. CI with this particular class. We went through demonstrated a workflow where we added a module with composer. I think of the address module because it's a great place to get started people use a lot so on a Drupal 8 site we added the address module composer. We pushed the branch that we had created for that up to github and create a pull request as part of that the build ran automatically with circle. CI it compiled the sites with composer pulled in all the dependencies kind of tested. It built it and placed this perfect version over. Pantheon pre-built tested and all that stuff ready to go and the visual regression test and all the other tests appeared in the pull requests automatically so all that the developer.
How'd you do use composer. Make the pull request. All the testing happened automatically and it shows up for their lead developer or their project manager. Whomever to verify that the things change that they're expecting the change if this is something that's interesting to you. I am expecting. We'll probably do this class sometime again. And we are right now talking about doing this as a either. A short webinar series like maybe three one-hour chunks or maybe a single three-hour training that we would do online. If you're interested maybe come up to me afterwards or shoot me an email or something. Let me know and we'll get you on the list for that. It's not something we charge for. So if like to learn more about this process let us know okay and then finally the standalone app so. I don't have time to go through this right now. Unfortunately but let me just talk about what it would look like so the scenario again is that you have a QA team. You have someone who doesn't feel comfortable running back stop gs themselves or configuring the mo file you can give them a anode app. That's pretty kind of set up pre-configured for your project so that every time that you run you know and node run start or whatever npm run start or whatever it might be that you run or as part of another operation you know maybe you put a button on their desktop and they click the button and it runs the test automatically and just pops up in their browser and they have a test that just ran because they they fired it off so all of these examples are here you can go download the examples. Run them on your own check it out see if you like it. The examples include the first one which is just it's just a site and you run it and it runs and it's done and then there's a multiple site example where you run it and ask you which of these three or four sites would you like to test you choose the one you want and then it goes and tests it and then you can run all in the next block but cool so any questions.
Yes hearts yeah so so. I was so in the mini sense that I've run this on this session and workshop that slideshow at the top gives me varying degrees of difficulty and I can't consistently figure out how to fix it every single time. I was fortunate this time that simply setting the delay was enough. I was able to say you know. Wait this long and then take a screenshot and then it doesn't catch it anymore occasionally. I'll have to remove it or hide it so for example. Let me just do that real quick and we'll see how that works so here back in my backstop. JSON file we have a section for yes. Thank you right here. Hide selectors so if we just put in a selector maybe in flex slider is the like service to be put in. Hi selectors like this and we go back. We're gonna have to do back. Stop reference to regenerate our reference files long rating for that looking at this so we have hide selectors which will just do a. It'll do a hide but kind of keep the element in place so nothing on the page should move. It'll just like blank out that area so it doesn't affect the test versus remove selectors which removes the code and everything moves to account for that thing missing from the page at that point so. I've found hide selectors to be a little better but sometimes you just need to remove something for other reasons hopeful that completed so backstop test. No so again with that scenario that random happenstance situation where my slides were different all that I did was go into this back step that JSON file to find my live slides versus my local slides. I did a test so I didn't have to set up a brand new project or do anything fancy. You don't have to use it for a website. You can use it for anything but as you can see we. We hid that slideshow element at the very top and oh I forgot to add the the hide element for the second scenario but for the first one. It's it's not there anymore.
Austin is these basically the same one you're really gonna see if the Sun of fame when you're running it on Thea VI so I it's pretty much the same thing. Yeah yeah it depends on exactly what you wanted to do. You're you're able to set up your backstop that JSON file so you can pass through like variables you know so I for example if you take a look at the code for the node app which you do if you follow the exercise you actually go in and you have a separate backstop file that is like sites to be tested and you define all the important elements for each site in a object or an array or something and then each of those is passed into a generic backstop that JSON file via with variables. And all those things so depending on what it is that you want to do and how complex you want to be you. Could you can do other things. But it's pretty similar to this the better performance next got some sight the see. I whipped the performance it for an extent to the fight with the eye. What do you mean if you brought a lot of pesticide what how are you. Haven't we like performance of the site or performance of the test. I mean you saw you know adding its it's cumulative or exponential but it like for each test it takes you know if each test takes five seconds or ten seconds. Each viewport multiplies that each scenario. You know so it. It grows pretty quickly. Yeah yeah and so in the interest of making it something you'll actually use and not be annoyed by you. Want to keep it pretty lean but if you have this as part of an automation step that you run every time that you do a pull request you don't want to forget anything and so it might run longer but you kind of forgive it because you just make the pull request and go get some coffee and it's not something you necessarily do every day all right well thank you all very much for attending and I hope you all the great rest of camera.