Hey, everyone. You know, I'm always encouraging you guys to use more video to fast track your improvement. And on today's episode, I have a short interview with Swing Vision CEO, Swap Nilsahai. I've been using SwingVision for a few years with my full-time students, but also with the students I work with remotely doing match reviews. On today's episode, we discuss what stats SwingVision can track and how its AI can help you make better tactical decisions.
so sit back relax and prepare to become a smarter tennis player all right swap no welcome to the pod yeah hey thanks for having me super excited to be here so what i've learned is as a coach uh i have maybe complicated or detailed things that I need to express to a player and they might be new to the game. So I've got to learn how to dumb down sometimes a complicated tactic or idea.
and explain to someone who doesn't really know much about the game to begin with. And so I love video. I love information. I love stats. I have no idea in the world how something like Swing Vision could even... work. So I would love for you to attempt to explain to me how this even gets created and how it even functions. Oh boy. Well, some background on myself too for... you know your your listeners or viewers who aren't familiar with swing vision
I was at Tesla before working in their autonomous driving team. And there I was working on specifically like tracking the cars and pedestrians that are around you using the cameras. So like every Tesla has like eight cameras around it.
There's these very advanced AI algorithms that you can train by looking at thousands and thousands of images of cars and pedestrians. And you can kind of basically teach this car to recognize important things. So I got really... experience in that working there um and then also been a lifelong tennis player and so i i wanted to be able to track stats and it's kind of stuff you're talking about you know just be able to get data on my game basically and kind of realize working at tesla that
You know, cameras could be an interesting way to solve that problem. Obviously very challenging, but it seems like maybe we could solve that problem. And so that's the approach we took, which was... Let's use your iPhone camera, first of all, so that it's super accessible. We don't need to install some crazy like 10 camera system like Haggai, which is very expensive. And just from that one camera... You can actually see the entire chord and you can see what everybody's doing. You can see...
What type of shot they're hitting? Are they making contact with the ball cleanly? Where is the ball landing? I mean, there's a lot of information that you can actually see and your brain is processing all this automatically when you're watching a tennis match or when you're playing tennis.
But basically anything that's visual that you can see, you should be able to train your phone to track it. And so that's the approach we took was, you know, let's go and collect like thousands of hours of tennis footage, YouTube, ATP WTA matches.
your local weekend warrior uploading his match on YouTube to his like 10 followers, whatever is out there, let's get all that. And then let's train a system to actually recognize like shots, like recognize when a forehand is hit, recognize when a backhand is hit. And then not only that, but let's actually recognize like, where is the ball right now? And where is it landing? And did it land in the service box? Was it in no man's land? Was it out? And so...
We basically went through a lot of the hard work to just collect lots and lots of data and lots of examples and annotated it very precisely in terms of... exactly when does a shot happen what kind of shot is it all these interesting details that we want to track and then we trained the system basically just on thousands and thousands of shots and at this point it's like millions and millions of shots it's really cool so
Now that Zoom Vision has been live on the App Store for about five years now, we have... almost 200 million shots that have been recorded on swing vision by a lot of players like hundreds of thousands of players around the world so we have such a diverse set of footage and we have people hitting in all kinds of different strokes and ways that maybe wouldn't be recommended
but it's in there in the footage and different lighting conditions and court services and like people setting up their camera in all different kinds of ways so we just have this really like amazing diverse set and so that makes the ai like really robust and it's able to like track all these things and what's really special at swing vision
is that we do it from one camera. So there are other solutions in the market, but we're the only one that does it with one. We have a patent for that. And it makes it the most affordable solution possible, essentially. So I told you before the show that I'm kind of on vacation and my wife got a Tesla from Turo. It's like some car rental thing because we're here for a while.
and i've heard this word before like friction right so i've never driven a tesla so i got in there i was like so annoyed i'm like i don't know how anything works what do i do how do i charge the battery and it was so uncomfortable for me i was like can we just give this back and get a normal car that i'm used to and i hear that a little bit because i give swing vision out to my campers when they come to my doubles camp and even just the idea of hanging that pole
and using some piece of technology can be intimidating for people and i'm assuming that setting the camera up correctly is vital to getting accurate information so what's the easiest or simplest or most efficient way for them to get that set up for their matches yeah i mean We spent a lot of time over the years recommending different options and seeing what sticks and what do people like using. And what we found in the feedback was like far and away.
having a sort of selfie stick type of device that can just hook onto the fence is the fastest because unlike a tripod, which has like three legs, this is essentially just one leg. It's a telescopic pole. I actually have it right here. And so, you know, this is the easiest way. It's called the swing stick.
And it has these little latches that you can open to... extend it and it just hooks onto the back of the fence right behind the baseline and it's really fast to set up and yeah i mean there is some friction the first time you use it you might be a little bit hesitant but once you get the hang of it it's it's really easy um and the nice thing is if you have like an apple watch
or your friend has a phone that connects to your phone, then you can check the camera so you can make sure it's set up properly. But that's really the best way. And that angle on top of the fence is so similar to what you'd see in a broadcast match.
And it lets you see the ball really clearly, regardless of which side of the net it's on. And so it allows you to get accurate stats on both sides of the net, which is really important if you're playing a match, especially because you're going to be switching sides and all that. So yeah, that's the best way. And then we have plans to improve that as well. So later this year, we're actually going to add a motor into the Sphinx Stick.
And so you'll just set it up and then it'll automatically like tilt the camera and find the court for you. Because that's probably the last kind of friction point is that you may not get the angle right the first time you put it up. So you have to kind of bring it back down and adjust it. So we're going to solve that problem.
but the new version that should be coming out later this year. So that's super exciting. That's incredible. So you were a tennis player growing up. You played, you still play recreationally. I'm actually very curious in your answer to this, but Swing Vision has lots of cool things that can track things that I'm interested in, your court position, where you're hitting the ball, what shots you're hitting from where. For you as a player, which one of those features excites you the most?
Yeah, I mean, I think this, yeah, it's just, it's so much that attracts. I think when you're... initially using the app it's so fun to just watch your video and see like the shot speed i feel like that's the first thing people see is like oh my serve is really that slow um they're a little disappointed but you know you get to see that and then but it's fun because you'll see like a really nice point and you hit a winner it's like wow
that was a nice like 60 mile an hour forehand or whatever it is which is high for an amateur i think um you know so i think that that's pretty fun to see initially it kind of grabs your attention and seeing where the ball is landing it's really cool and like very sci-fi but i think personally for me
I really just like watching the video and looking at specific shots that I hit. So what's nice with Swing Vision is you can filter the video. And so this happens to everybody, but like, you know, you might have one match where you're just like, oh man, my serve was really off. Or like, it was just that second serve from like the ad side.
i kept missing it or whatever it is and so it's really easy with swing vision to filter that you can say show me every second serve from the outside show me every second serve from the outside that went in the net and it'll do that for you and it'll just show you those shots so
I will do that really often after a match where, especially if I lost, I'm trying to understand what was going wrong. And usually I know during the match that like something's off with one of my shots. It's really hard to like correct in real time sometimes. But being able to watch it afterwards is so much better. And you can see right away like, oh, this is why because the toss is wrong or whatever it is. So like, I think the video review and the fact that we have those filters.
is my personal favorite thing um and i love being able to use that to just like look at my game and it's it's it just gets so deep like you can look at like your return from um deuce side specifically your backhand return i mean it gets really deep and so like whatever thing you're struggling with on the court you can just look at that right after
And it's just like the fastest way to improve, I think, because it's like, okay, that's what was wrong. Next time, I'm going to just keep that in mind. Is there a way, and maybe you guys already do this, or a way that AI can do it in the future, but tying in the stat to if you actually won the point. So, for example.
one thing my players are super sensitive about is they never want to hit short right and i agree like hitting deeper is objectively better than hitting short guess what hitting short in the court is a lot better than missing deep so there is like a balance there and a lot of times
People don't hit approach shots well, they don't volley well. And so I'm like, hey, hitting short is not the end of the world. And so when I do match reviews with them, I usually show them all the returns, all the short ones and go, look, it wasn't that scary. Does swing vision have a way to also like kind of plot those things together and go like, here are the returns on the points you won, or is that something that could ever come out in the future?
Yeah, that's definitely something we could add. I don't believe we have that specifically today, but you can look at points that you won, and then you can look at specific shots based on the score. So show me all the returns.
on like 1530 points or something like that, or the 1540 points. You might be interested to see that or just break points in general. And so that's really helpful. But yeah, it's pretty straightforward actually to add that additional layer on top, which is like show me all the returns on the points that I won versus the ones I lost.
um that's yeah actually pretty straightforward to do so if more people request it we'll definitely add it right i i got a lot of requests for you like behind the scenes i'll be emailing because i love all this stuff um one thing i have a theory Well, not a theory. I actually think feel like it's pretty proven is that especially at the amateur level on big points, like you just said, you can do 30 alls, 40 alls, no ads, whatever that.
Most of those points are lost, not won. Just because errors are a big part of tennis and then under pressure, most people are not elevating their game. They tend to drop their level, right? And so... Have you noticed, I actually emailed your people and they said you guys obviously have millions and millions of points.
When you guys get all that information, is there anything that you guys do with that where you can see that as a collective for like all of amateur tennis where you say, hey, this is something that. It tends to happen for people that use SwingVision, or do those points only just help the app and the AI be more effective and efficient? Yeah, so today we're using it.
mainly in the context of an individual match. We do do some comparisons, like match over match. You can look at some stats and how they improved. But we've really gone deep right now on an individual match basis. But we're setting up the infrastructure this year so that we can... do more of this comparison across several matches across several different users or maybe even like categories of users like look at everybody who's like a 4.5 or a 4.0 and like
compare them um so yeah there's been some kind of infrastructure work we have to do to like make those data points easy to to like generate because it is a lot of data to sift through so i think That's the hard part right now is that it's kind of slow. We could do it, but it's just really slow. And I'm hoping we get to a point where we can share some really cool reports or infographics, just general player tendencies that we see.
Here's what 3.5s do really differently from like a 4.0. And I think that would be so helpful for players to understand. And we have the best data for that. So yeah, definitely lots more planned there in the future, I would say. But today in the app, it's more kind of individual match focus, I'd say. um but we're trying to lay the groundwork for that right so for each player right now they could for example like core position is a massive thing so they could look at you know
From five matches ago, they can see their spray chart. Here's where I was hitting my forehands from. I wanted to get more in the ad court. I wanted to get more opportunities to hit inside the baseline. And then they can compare that, say, from a match that was a week ago. So they could say, hey, in the last three months, I can see. My core position is improved by three, four feet, and I am getting more forehands from the ad court.
Yeah, exactly. And then you can do similar things for like your depth of your shots, like my return depth, you know, what percent we're going like beyond the service line or something. Right. So you can see that and compare that from match to match. So that's that's really helpful. And that's like another thing I like to use where. if i know like there's something specific i'm working on then i'll
That's where I'll actually look at the stats. Like I think in general, I'm using personally like more of the video review, but the stats is really helpful. I think when you already go into a match, knowing that you're trying to work on something or go into like a practice session, knowing that you're going to work on something, because then you can actually look out for that stat.
and see like how it compared to the last time you were on the court. And that's just so helpful to be able to see like whether you actually made that improvement. And then you're also like more intentional about like trying to increase that score sort of makes it more like a game, right? So it's kind of interesting. Is there any element of the app or...
all the possible benefits that SwingVision brings. Like I said, I love video. I love all this. So hopefully I would know all of it. But is there anything that you think users out there are underutilizing that can help their game with the app? Yeah, I mean, I think the biggest one, which is... Kind of a more new feature that we released last year is this concept of giving you actual strategy recommendations, which is really cool. So we call it AI Coach.
But it's not replacing a coach or anything, but it's kind of cool because it does give you this like strategy recommendation. But what it'll do is it'll analyze your match, all the points won and lost, and then it'll find what patterns were most successful. And so if you've watched broadcast TV, like you've... You've probably seen something like this where they will show it to the IBM Insights and that kind of thing. But it's a really, really useful tool. So for example, like...
I was looking at a match. I'm going back up to the Bay Area to meet my team, and there's one engineer on our team that I always like to play singles with because we both have one-handed backhands and we're very similar playing styles. So it's just a fun match with good highlights.
So I haven't played him in a while, but I was going back and looking at his match, and it was a match that I actually lost, but I was looking back and seeing what were my most successful strategies. And so it'll show you when I was serving from the deuce side.
um if i served out wide to his forehand and then hit it back to his forehand that was actually my most successful strategy out of every possible strategy that i had so it's kind of interesting so you can see for that particular matchup like what is working really well and then um conversely you can also see what was contributing to the most points loss so like the out of all the different combinations of shots that happened like i lost the most points when
um he hit it to my forehand and i went for like a down the line forehand like that was how most of my points i lost ended that way um and obviously there's so many different ways that points get ended so it's not like a really large number but it is the largest one and so it's kind of interesting to see that i think that's
That's actually really helpful. It's very underutilized. It's called the AI coach feature. So if you have your match and it gets like automatically scored for you by Swing Vision. we also are able to generate this strategy breakdown. And not only does it show you the strategy or the issue, there's a play button. You can actually play all the points just corresponding to that.
kind of strategy. And so then you get the visual feedback as well. Is there a limit to, so you said like, oh, I serve out wide and then you took a ball behind him back to his forehand on the deuce court. Is it a two shot limit, a three shot limit? Cause this is actually something I was going to ask later.
which I'm super interested in is the setup of the point. A lot of people, they might play the point perfectly and then they miss the last shot and they go, oh, my point sucked. But in reality, they actually played a really good point or they could have played a horrible point and for some reason they get lucky at the last second.
Is there a two or a three shot limit to that AI coach? Because that sounds super fascinating. Yeah, right now, because it's just using the data from an individual match, we've tried to focus it on like maximum two shots back to back because. Once you get to three, then you start to get like really small sample sizes, especially at the amateur level, because points are so quick, right? But I think that, again, as we build infrastructure to be able to...
generate this data across multiple matches, then it starts to get really cool because then I could look at my last 10 matches against this person or five matches and it could actually have enough of a sample size to show something interesting about like a three or four shot pattern. But yeah, right now it's... mainly focus on kind of the first two shots on serve the first two shots on return and then also just any other like sequences of shots that that are resulting in errors basically
So AI coach sounds awesome. I didn't know about that. Are there any other new-ish features in the last couple of months or features coming out in 2025 that you can tell us about? Yeah, the most exciting one recently, which I'm using every single time I play now, is our new live line calling feature. So, Singusion has already been kind of famous for providing line challenges where... When you're playing on the court, you can pull up a line challenge from your Apple Watch.
And it's really cool because you can just say like challenge and then it'll actually show you like a slow motion replay and it'll tell you whether the ball is in or out. So it gives that decision for you. And that's gotten just so much more accurate over the years as we've gotten so much more data. And to the point where last year... We were checking the numbers on this. We actually officiated 500 matches, USTA matches last year with Swing Vision. It was crazy. So we built a whole new solution.
where every court at a tournament would have an iPad at the net post, and players could walk up to this iPad and challenge calls. And this is all done by USTA staff, specifically here in Southern California. And so there were some tournaments where we were on like 12 courts simultaneously. Swing Vision was on every court. Every single kid had the ability to challenge line calls during their match. It was amazing. And so players were still calling the lines themselves, but...
If they doubted their opponent's call, then they could go to the iPad and challenge it. And people just accepted the result and moved on. It was just, it was so amazing to see like the positive feedback and just people were less stressed and less worried about hooking and all that stuff. And so it was a really, really great experience. And so...
That's been a big effort of ours in general has been like bringing line calling to amateur tennis, starting at the junior level, because we want to try to get people where they're most likely to drop out of the sport. We want to make sure they stay in the sport. But, you know, eventually you could see us bringing this.
to adult leagues. And so that, that was a big, big effort that we did last year. And there were some major improvements in our line calling accuracy that allowed us to provide that experience. So a lot of the, a lot of the hooks out there or the people that would be upset. would want to know what you think the accuracy is with those line calls yeah so um we i should spend a lot of time validating the line calls and what we'll do is
we'll set up that one swing vision camera, but then we'll put like 16 really high speed cameras all around the court to like know exactly what happened in every possible scenario. And so we kind of validated against that. So it's almost like comparing swing vision to like a Hawkeye system, but it's even more than that.
that actually and so yeah through that system we verified that swing vision is more than 99% accurate in general if you've set up properly on top of the fence and everything and then what's really cool is If the ball lands within 10 centimeters of line, so it's like a really close call, then swing mission is 97% accurate.
as a reference point the human eye is only around like 90 to 92 percent accurate for that kind of close call so it's surpassed the human eye for close calls at this point which is really cool to see and it will happen to me so regularly now where i'll play a match
And I'll be like, oh, I'm pretty sure that server is out. Let's challenge it. And it's like, nope, it was right on the line and you can see it in slow motion. And it's because AI essentially sees everything in slow motion because it's processing it 60 times per second. So, you know, the ball moves very slowly for the AI.
your eye, it's moving really quickly. And so, yeah, so it's at that point now where it's kind of taken over the humans, which is kind of funny. But the new feature, which makes this even more exciting, is that... we have this new live line call feature where
Instead of having to even go to your watch to challenge, your phone will just play a sound as soon as the ball bounces out. So it's exactly the same experience you're used to seeing on TV now. Pretty much all the grandsons have moved to this now. It's called Hawkeye Live, where you'll hear that. You hear the outs.
sound and it sounds like someone's saying out but it's actually just computer generated and there's no linesman on the court anymore right and so we've basically brought that experience to the to the amateur now using just one phone. And so this has been super fun to use when I'm just playing with friends or rallying around and you just wait to hear that little beep from the phone and you know that it's out. And again, it's just so good. Like it's so accurate.
period. And so that's been really fun to use. And it's only going to get better and better. And I hope that eventually we'll bring that to junior tournaments as well, the same thing. So instead of players having to call the lines, they won't have to do that anymore. They won't have to be worried about that. The system just doesn't.
for them and they can just focus on playing so i think like that's our long-term vision with that feature but it's not quite ready for prime time but it is available to consumers if you want to use it you know just just for fun when you're playing with your friends
Right. Vic Braden, I think he did the example where he had like people jog around the court and try to read the newspaper. And it's like, you can't read anything. Like even if the font was like 72 font size, like you still can't read it because when you're running. Your eyes don't work as well. And then they're just human eyes anyway. And then on top of that, lots of players in leagues have bad intentions to go with that.
It's funny because I would love that for juniors. It's a thing that I get messages about all the time is how do I handle cheaters? What do I do? Blah, blah, blah. And even if the app is 99%, even if it was 95% accurate, it's an unbiased.
It's not sitting there going, I care who wins. It couldn't care less. And so at least it's going to be equal. And I've heard that from pros as well. Like with Hawkeye, I think I saw, I can't remember who it was first round at the Australian Open. Maybe it was Osaka, but it looked like the Hawkeye missed a call. and it's like okay like that stinks but almost every single call is accurate and there's not a human being influenced in the match and there's no bad intentions so
In that sense, it's a little more fair. And I think that would clean up the game and people wouldn't have to worry so much about line calls. They could just worry about enjoying themselves and playing better. Yeah, 100%. And that was the approach that USTS SoCal took with it too, which is like...
We're not looking for perfection. We just want improvement and it's way better than the human eye. And like, that's basically the bar, you know? So, um, yeah, definitely. I always, if I'm doing a tactical video, A lot of times on my podcast, I'm encouraging people to video their matches. And it's like, you know, telling a kid to eat their vegetables. Like I know, like whatever.
5% of them are going to go do it. What is your best pitch as not only the creator of Swing Vision, but as a tennis player yourself who's using video to encourage people out there to take that extra effort to get past that initial friction and start videoing their matches and have more data?
Honestly, I would say for me, even if you're very serious and competitive or you're just playing casual, I think the one universal feature that everybody loves and the thing that... makes me coming back to swing mission every time is the highlights feature and the fact that it's finding those like amazing memories and moments that you had on the court like that to me just like
Every single time I play, I'll go afterwards and look at those highlights of just those best points, the really funny points that we had. It makes it so easy because you can use your Apple Watch to just bookmark the point, just one button, bookmark any rally.
And that to me is like just a way to relive those moments. And like, I can go back and look at a match now from like five years ago and see that amazing like backhand down the line winner that I hit against that one friend of mine. And it's just, you can really relive that forever. And if you don't.
film that it's gone you just play and it's gone and it's just in your in your memory and that's all you have so i feel like that's the one reason people should use swing vision or just record the game is like you will have this footage forever and like you can share it with your friends and family you can send it to them not everyone's going to go out to every
match that you play unless you're you know a top player right so it's like really nice to go just send those highlights and i'll do it all the time i'll send highlights to my parents um i just hit with my wife last night and we were just watching on the tv screen like on a bigger screen our highlights we had some really funny points and
It's just a great way to relive your tennis experience, I feel like, and share it with others. And that to me is just amazing. I feel like it's a paradigm shift because... We've been playing tennis for, you know, it's such an old sport. It's like over a century old. And there was no way to even like capture or share like what was happening. And now it's all there and it's all there forever. You can share it with your kids someday, like whatever you want to do.
So I feel like that's a really good reason. But then obviously, if you're really serious, then you can go further and improve your game. And there's no better way than just watching your game. And it is the fastest way to improve. Because you can just figure it out so quickly what's wrong and get ready for the next time. I'm sure you're very busy, but is there any way you can write me some code where my highlights can come out as the most mundane average? Because as a coach...
I do love, like, I want them to be motivated and excited, but I'm also like, can you please not try that highlight shot? So I want, I want, I want you to write me code of like, Hey, here are all the simple forehand cross courts you hit that won you a point because the other person made a mistake for no reason. Like that's game. You think you can. You could write me that code. I'll work on it. I'll check with the team.
Amazing. All right, man. Well, I really appreciate you coming on. Like I said, I've been using this for a couple of years. I give it to my campers. It's how I coach people remotely. It's an awesome product. I really hope people use it, video their matches. And like I said, I know you're a busy man. So thanks so much for coming on. Now, of course, thank you for having me. Super fun. And yeah, thank you for all your support as well as always.
All right, I want to thank Swapnoll for coming on the pod. Like I said, this is an app I've been using for years, and whether you guys end up using SwingVision or something else, I highly recommend you video yourself playing matches. It is so important to the learning and growth process, so if you're going to invest your time and energy into the sport, you might as well maximize your improvement. As always, thanks for listening. I hope you just improved your tennis without even hitting a ball.