If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Beah: Hello, and welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering approach. And in this particular episode, you’re going to hear from Mihai, who is here to talk about the various UX improvements that we’ve made to Duck.ai recently. Mihai is on the design team. And do you want to just give your give a quick introduction of yourself, Mihai?

Mihai: Sure, thank thanks for having me. My name is Mihai. Originally from Moldova, lived across the world in many places, now based in Switzerland, joined DuckDuckGo maybe around three years ago, and I’ve been on the Duck.ai team almost from the very beginning, so I’ve seen the product evolve since then dramatically. So so yeah, we’ll talk about it today.

Beah: Nice. Well thanks for joining. For anyone who hasn’t met me, I’m Beah. I’m on the product team here. I’ve been around for a while. So yeah, I’m gonna go ahead and ask Mihai some questions about his work on Duck.ai and we’ll just dive on in. So I think I said we’re talking about improving Duck.ai UX, and I’m not sure I said that if you’re not familiar, UX is just a fancy abbreviation for user experience, even though we all know experience doesn’t start with an X. So this means like, you know, making the features that we have and the functionality that we have more usable, tweaking the functionality to better like suit people’s needs and desires, and just make the product more delightful and easier to use. You feel free to amend that definition. But just to kind of like set

Mihai: Feel free to amend definition.

Beah: the set the tone a little bit, Mihai, tell me a little bit about like how we even decide to work on something. Like how do we prioritize UX user experience changes in Duck.ai land?

Mihai: Yeah, as designers, we’re basically I I tend to think of myself as the diplomats that we we try to talk to as many people to get our work done. But we use various signals to inform these decisions. Some of some of them are more perfect, some of them are imperfect. There’s no There’s no reality where we’re working with perfect information. So sometimes we form assumptions, sometimes we rely on research. User feedback, internal feedback. Sometimes we have strategic bets that we place as an organization. We have dashboards with metrics. We have just no we know the competitive gaps that we have, and sometimes we just rely on instinct. So we

Beah: That’s good.

Mihai: we bundle all of these together in our decision-making process. Some of these signals they surface a problem,

Beah: Yeah.

Mihai: others validate the direction and it’s on us to use our own human judgment to determine the right trade-off for for these things. And in the end it narrows down to to looking at where the biggest overlap is with the user problems that we have and where we’re taking the product.

Beah: Yeah, or do all the hand and just then follow it.

Mihai: Yeah, maybe this is more kind of like general response, but no one decision was the same.

Beah: Yeah, right once. Yeah, yeah.

Mihai: So it’s it’s it’s all based on experienced people in the room making the the calls based on the information that we have.

Beah: Yeah, I mean one thing that I’m guessing people I I bet a lot of people would underestimate the degree to which we’re obsessed with user feedback. Like you can’t completely rely on user feedback to tell you how to build a product because, you know, hopefully the people building the product have additional insights about what’s possible and what good looks like that users can’t articulate or shouldn’t take the time to articulate. It’s not on them. But that aside, like we obsess pretty hard on feedback. Like we read an insane amount of feedback. We have systems for categorizing it and making sense of it. I mean, I’ve been caught up in like long discussions where we are really deeply trying to interpret a comment somebody made on Reddit or in the app store and like try to figure out the best way to address it. And so I’m just pointing that out because I think that might not be obvious. Like I think there’s probably a lot of, you know, for a tech company that serves millions and millions of users, I’m not sure that’s I’m guessing that’s not the norm. So that is a big part of it. But then like beyond that, you know, there’s a bunch of other signals we use, like Mihai said, and I know in Duck.ai on that team, like you guys have a rubric, right, that you’re using to actually like evaluate in a structured way a bunch of project ideas and feedback is one of the inputs to that rubric, but there are others as well. So also very maybe distinctive of DuckDuckGo is I think it’s a hyper structured and rational decision making process. Would you agree with that, Mihai?

Mihai: Yeah. It’s it’s quite unique and I think the the great thing is that a bunch of us, everyone here, they’re scoping their own projects, they have their own ideas, they uncover different things that they’re pitching in and that goes into that rubric. And at certain times we those categories of how how we score the rubric, they’re weighted. So sometimes certain elements are more important than the others. And that help us kind of look at what what’s important to us right now and what we’re getting from the users and all the projects that are strong candidates for us to tackle tackle in the next roadmap, for example.

Beah: Cool. Yeah, so let’s talk about something that did bubble to the top recently. Like, can you give me an example of an a UX improvement that you’ve worked on?

Mihai: Yes, so the first example is dictation. I’ll share my screen real quick. Cool, so this is our interface here. So users keep asking for dictation features, so it’s this microphone in the input field. There’s certain people that for them is they would use the input field. They’ll rather talk than type. It’s either a prompt is really long, or you they just want to do it hands free. And shipping this was actually great because it it got us in line with the competitors as well. But internally we we were kind of i it it was the group was split in terms of like if we wanna do it until that user feedback kept coming back, kept coming back. And we decided to to pull the trigger. For from the UX standpoint, what was exciting for me personally is voice interfaces, they have a big affordance problem. That it’s really hard sometimes to know what’s the system status for that and to avoid the jargon is basically you need to know what the what the app is doing at any point in time. And that was an interesting challenge on top of just delivering the feature. And if I if I trigger it, we we do three things here. We label things clearly for people to know what’s going on. You also get a visual cue, this waveform that provides feedback, and as you see me pause, that waveform reacts to my voice, but also the input field has changed its its shape not the height or anything but what’s inside it now is geared toward towards a different input which is not text and we’ve added this nice glow animation to it which is which is really cool so and then it’s trust transcribing and giving back the the text. Another one should should I should I give a few more? Alright.

Beah: Nice.

Beah: Yeah, yeah. Give me another example, please. That’s good.

Mihai: Alright. Another one is batch deletion, which is another feature that was requested by by users. This one is a little different because our we’re famous for our fire button. You you can basically burn all your data and stuff like that. But in Duck.ai specifically we had the reality that you could either delete one chat at a time or all of them. And nothing nothing in between. And we we decided to that we need to give people more precise control. To do it in just one step. Whenever you have 30 chats, for example, and you just want to delete seven at a time, it’s quite annoying to go one by one. So this one, it’s not groundbreaking or anything. It’s been a pattern on the web for quite some time. How you can batch select the different things. This is a great example, thinking back to the signals that we spoke. It’s a great example when you need to know where to follow the process for something when whenever you’re designing something that’s more unknown and whenever you need to trust your instincts more. And this is a project that was basically that where we we used our experience more and the project was short this was delivered and and shipped quite quickly because of that. And what the feature does you can go in and select many chats you want to delete. We have different entry points to how to do it. Usually

Beah: Yeah.

Mihai: we know people click the fire button to open it so we surface a button there for people to actually not delete everything but select multiple ones or for maybe power users whenever something’s selected you can shift click and enter this mode and delete delete your chats. So yeah, this one the reason I’m I’m asking is we spoke about all these signals and how complex things are, but this one was th three designers, we got in the room, we did a thirty minute thing, then I spent maybe two hours on it on edge cases, we put it to review to a wider group of stakeholders and it went straight into implementation, which is which is great because we have features that are a little bit more complex which require all all of that

Beah: That’ll be true.

Mihai: kind of digging, research and all those other things. But this is a great outlier maybe where we

Beah: Yeah.

Mihai: can just roll with differently a little bit.

Beah: Yeah, yeah. Also, there’s yeah, I j while you were talking, I was thinking about we ha we also added not too long ago another another way to delete chat level sort of at your cadence, which is like when you hover do you wanna show your screen again real quick, Mihai? When you hover over each chat in your chat history, you get a fire button. So you can also just tap that. So if you want to do one at a time, but you know, from a list, that’s another way to do it. I think that was a nice win. And then I don’t know, just for a little like real time inside baseball, well, truly an hour ago, Mihai and I were on a a thread where we were talking about with some other folks whether what the equivalent best equivalent of that on mobile is where you don’t have hover, you know. So we just have to if we want I I personally love and use those that per chat fire button. You’re like, I don’t need that, I don’t need that, I’m gonna clean it up real quick. On mobile we don’t have that. So yeah, like is it worth the real estate to just always show the fire button? Is that too noisy or people gonna misclick it? So we were kind of debating that this morning and gonna get it into an internal build and like kind of feel it out ourselves and then you know if it feels good we’ll we’ll release it. But we’ll watch any feedback coming in as well to help guide us on whether it’s working as we we hoped it would. So hot off the process. Alright, do you wanna do you wanna talk about one more we talked about maybe the the model picker might be interesting to show ‘cause that was a nice and nuanced recent UX improvement.

Mihai: Yes, yeah, let me share my screen again. Yeah, this one this one is not the one I designed, so I don’t want to upset the designer who actually done the hard work. But

Beah: Mm-hmm.

Mihai: I I advised this project so I stayed close to it. So I’ll it’s it’s Esteban. So thank you Esteban for

Beah: You wanna give him a you wanna name drop? Give him a shout out.

Mihai: a better model thinker. We’re all grateful for for that. Yes, it’s basically our our team has grown, so it’s it’s great that we we can cover more ground. And Esteban is not new on the team, but whenever whenever I started it was just me. So it’s great to see all of these things basically. Ship and all these amazing features that We can put out much faster. Now this one is basically speaking of old stuff, the old mode model picker had a whole lot more details in in in it, and through research we’ve uncovered that people just could not understand what’s in there. And they were stalling, trying to make sense of it a little bit. Power users have no problem, but I don’t know like general public, people who are actually not experts on just just want to use the tool. It it was a thing that was giving them pause. An another thing that we’ve uncovered which was useful insight is once they got past that confusion, they really valued it and it was the thing that they would recommend

Beah: Yeah.

Mihai: to friends and stuff. And on top of it we also as a company, Duck.ai specifically, we provide all these L LMs from different providers. So this is Some sort of a our own differentiator where you get the private versions of all of these. So we wanted to fix this old problem of people stalling and being a little bit confused about it. And the solution was basically again based on insights, we we realized that much people knew what they wanted to use or had some general idea and they they

Beah: Again, you know, I we like that people looking on a TV.

Mihai: just use what’s giving them a good response. They just stick to it and majority don’t really switch. There are use cases for some that that switch for different particular reasons, but for this particular model or now a drop-down they just needed something much simpler and the job of this is to show that there is choice because we provide all of these things, variety, but also we wanted to remove the guesswork for for people. So

Beah: And then you’re going to

Mihai: Esteban has done an amazing job of condensing this into showing that variety. But we switched what we are doing here to for the user is we’re providing now a recommendation of what we think and what we think that they’ll use to have a great time and a great experience on on the platform. For each subscription tier we have Free, Plus and Pro. We recommend two models, one always good for everyday use. Based on their tier and another one which will give even a maybe a stronger, more comprehensive response, but will use more limits or it’ll it’ll exhaust those limits faster basically. So yeah now it’s kind of it’s much simpler and much more enjoyable at least for for for me and

Beah: Yeah. I mean

Mihai: based on your the research for users to use.

Beah: Yeah, one detail is like before it was a button that you pressed and it opened a modal. Like this it opened a new screen effectively with the with your choices and then you select your choice and then click start new chat. And like even just it’s a small thing, but like the the difference between what feels like kind of a drop down and something that feels like a new screen where you have to pick and then say start new chat, like I I I knew how it worked, and it’s not like one click takes so so much time, but the sort of like cognitive friction of like picking and then and then clicking something again, like start new chat, and like usually I hadn’t even started a chat. Like it just it didn’t feel easy peasy to switch around, you know? It felt like higher friction than you needed to so like you everything else is that even just kind of moving that like changing the UX pattern, I think is a nice, a nice little win. Sweet. But that’s one that probably is like counter to your batch delete example. Like we really did kind of agonize over a bunch of nuances because like with any design decision there’s trade-offs and there’s different cases that you have to account for in different states and like it did it took a while to to reason through it all. Speaking of which, can you give me, Mihai, an example of like a difficult or non-obvious design decision that you’ve had to make recently?

Mihai: A recent one. We always I think as a designer here I o I’m always reminded of how little we know about our user in comparison

Beah: Yes.

Mihai: to everything else we’ve experienced. I remember like we just don’t know the count of how many users we have just because we don’t track them. And stuff like that. Everything is very much on those guesstimates and kind of like I’m people sometimes use phrases like I’m confident in this assumption, like eighty percent and stuff like that. They give a percentage and it’s it’s a wonderful constraint because it’s it’s why I joined. I’ve recently so

Beah: Because it just to like because it’s privacy respecting is what you’re getting at. Yeah, we don’t track.

Mihai: Yeah. Exactly. So and and that goes with everything we touch whenever we’re designing it. It’s you you design something and it then then it’s through that lens of of respecting the the data and and providing a a great experience without needing all of those other things which other companies do. And I maybe don’t have a very good example to give that is recent one that is maybe an older one is whenever you’re getting a response how you’re switching through different models which is something Which is currently being worked on and where you will be evolving that feature soon. But that one, what wasn’t too obvious about it is that switching models is not a new paradigm in user’s mind because this AI thing is somewhat new. That’s why it takes the shape of a chat interface because it’s familiar. And switching a model in other places we’ve seen very much is has different pagination approaches and stuff like that, which is now learned but not natural based on what we’ve uncovered. And every time when I spoke to people, like I’ve seen so many confused face faces about like the pagination of answers and stuff like that. And what

Beah: Button on the natural work is everything natural.

Mihai: What we did, which wasn’t obvious obvious, we basically w took a more linear approach to how those responses are getting surfaced, which feels more natural into a chat interface, a thing that we’ve been using for decades. So

Beah: The patterns are a thing that we have.

Mihai: we kind of went against the grain there. And it wasn’t obvious at first.

Beah: So what would a paginated response look like? Like if we went that route, we would have like what, opened a whole new chat or something?

Mihai: It it no it it creates the same response but it gives you like a counter, like a page for the responses and then you paginate and it gets forked in different streams, almost like a decision tree that you can paginate back and continue conversation in different so that that

Beah: Yeah.

Mihai: even like I spoke to engineers who use AI, like they use Cursor, they use all these like advanced tools. And for some it wasn’t apparent that to to some that this feature worked that s certain way. So even at advanced people who are used to these tools were a little confused by the conventional norms, so to speak. So it made me feel really proud a little a little bit because we kind of took a different approach. And I think we get more than 300K uses on week from that switcher, which which is cool to see.

Beah: And three... coming.

Beah: Nice. Alright. Last question, Mihai, before we sign off, can you preview any upcoming improvements that people should look forward to?

Mihai: It’s the project I just wrapped, which is still being worked on. So the future may still evolve, but one thing we’re trying to add to to the platform is are you seeing my Figma, Beah?

Beah: Yeah, I can see.

Mihai: Okay, perfect. So this is very very rough. I think my is hiding my some of it. Basically we wanna add the share functionality to your responses, something that we we don’t do currently and we wanna do this at the response level but also at the entire conversation level. And once you once you click share, you’re basically sharing a response. You create a public link. That we’ll we’ll leave on our encrypted server with the encryption keys and all those other things that engineers are working on to make that super secure and everything. And then you can copy and and send it to to to people to to get that that response. And you’ll be able to to see it’s as someone who opens the link, you’ll be able to see what was shared and you you’ll be able to delineate what’s yours if you choose to reply or if you choose to add it to your own chats. So this is how it was designed. Sometimes we evolve the work as we play with it live when it’s when it’s in code. But it’s something that yeah should help a lot of people send a link as opposed to send a bunch of text.

Beah: Nice. That eggshell white is looking pretty sharp. Is that what it’s called, eggshell?

Mihai: It’s actual actual TW10 specifically.

Beah: Ooh, eggshell 10. Nice. Awesome. Alright. Thank you, Mihai. Anything you want to add before we before we close this out?

Mihai: This is great. I yeah, this is great. Thank you for having me.

Beah: Yeah, thanks for being on Duck Tales. Alright. Thanks, Mihai. Thanks everyone.

If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Beah: Hello and welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, the technology, and the people who help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering, or approach to AI. Today, you are going to hear about how we work, I guess, or one facet of how we work, which is using operational defaults and I have with me Marc, who is going to who I’m going to interview about operational defaults. Marc, do you wanna introduce yourself?

Marc: Yeah. Marc, I’m on the engineering team at DuckDuckGo.

Beah: Okay. That’s about all you need to know. and I’m Beah, I’m on the product team if if you haven’t seen me before. okay, cool. Well let’s jump in. What even is an operational default?

Marc: Yeah. Yeah, so defaults are I would say a deceptively simple concept. at at DuckDuckGo we’re always trying to leverage mental models to help us move more quickly and think more clearly and and things of that nature. in that context, the default effect is seen as a negative, right? And and I think we’ve all seen that in apps and in different places. Here we’re trying to leverage the the upside of that, where we’re basically taking any decision that you can make ahead of time that generally is true or has a right answer eighty percent of the time, we’re gonna call that a default. for that other twenty percent, we wanna leave room for our folks to be able to make exceptions, ideally very thoughtful exceptions.

Beah: When you say default is a negative in the app, you mean like, you know, if you get an Android phone and the default browser is Chrome and the default search engine is Google and it’s really hard for another player like DuckDuckGo to to break through th those defaults. Is that right? Yeah. Okay.

Marc: yeah, exactly. There’s there’s in Gabe’s book of mental models there’s the default effect and again it’s generally negative, but like I said, in this case we’re trying to use defaults as as something that we can leverage to help us go quicker.

Beah: Yeah, yeah. So how is that how is it different than a rule?

Marc: I mean it’s conceptually the same thing. I think rules are just more binary, right? Like this is something you do or you don’t do pretty much all the time. Where here we’re saying that these are things that are generally true. Like I said, eighty percent of the time is just the number that we made up. and that allows people to potentially make faster decisions. but we also want to include encourage people to think about what they’re doing and not just go on autopilot. And I think that’s the danger of catching something as a rule is that yeah, people will just kind of blindly follow it by default.

Beah: Yeah, yeah. I think that distinction is super important. Like, first of all, there are a lot of things where rules just have exceptions and therefore they’re not really good rules, you know? like you have to be pretty sure that there has to be a really good reason to make something a rule because it’s going to fit different situations differently. And yeah, it’s like once it becomes a rule, people like lose power to do anything differently, so they don’t think about it, they don’t feel empowered, and like with a default, there’s still a burden of judgment because you can overturn the default and choose a different path. but the idea, I think, would you care would you say this is a fair characterization is like A, don’t spend brain power thinking about like reinventing the wheel or thinking about something that doesn’t really warrant brain power, like there’s a good default, and B you know just to n to nudge people in the direction of the best average answer, you know.

Marc: Yeah, right. It’s leveraging decisions that we’ve deliberated over in the past and you know have come to a good decision on. and exactly what you said. We don’t want folks reinventing the wheel. That said, we do want I mean, we hire really smart people at DuckDuckGo and we wanna, you know, leverage and celebrate that. So yeah, everybody’s encouraged to challenge the norm, so to speak, or you know, pre existing defaults. And in fact that’s a big big part of this. whole concept is coming back and challenging these at times. But yeah, th they’re, you know, i in a lot of cases it’s it would just be wasted effort to be rethinking this from scratch. And as much as there are some that are that are rule or that you know might make better rules because they are more binary, there are others that really don’t make great defaults either. we had a decent example where it was like, yeah, there is something that was true, say fifty percent of the time. I don’t remember what the actual number was, but it just didn’t make a great default. I honestly don’t remember what it is, but yeah, d just to say that not everything c falls into that framework framework.

Beah: Yeah, that that’s a good segue. Yeah. T t give me some examples of things that we do think are good operational defaults or that we treat as such.

Marc: Sure, yeah. I mean we have a bunch I you know, one of the things that’s so powerful about this is that I think we all do this all the time. so for example at DuckDuckGo we try to prioritize quick wins, which I guess the other thing I should say about defaults is that they are somewhat self evident, like to the point where that’s why I said they’re kind of deceptively simple because there are things that you don’t necessarily feel like you should have to put a lot of time into, a a lot of thought into. But quick wins are one where, you know, we at the company we’ve been, you know, trying to push people toward that. Right. Like the idea of a quick win is something that is relatively low effort but has high impact. And I mean you would saying it out loud, you would say, of course that’s something we should prioritize. But I think it’s also important to acknowledge that people don’t think that way. instinctually in a lot of cases. They wanna sometimes they want to work on, you know, the hardest thing or what they see as the most impact when, yeah, maybe less would do. so that’s one. one of my favorite ones, and I think a lot of people at this company would agree is we do no meetings on Wednesdays. so that’s a very simple one that, yeah, you we we don’t I I I w wouldn’t call it a rule, whereas we don’t you know, you can have a meeting on a Wednesday and sometimes you have to, especially as if it’s with somebody external. But generally you shouldn’t be scheduling meetings on Wednesdays.

Beah: See, that’s interesting because I would have called that one a rule. We I specifically I would have called the rule no

Marc: I mean

Beah: If you want to hop on real time and talk to somebody internally about something as it comes up, fine. If you have like an external call, we’ve also accepted that. But I I try to enforce that one as a rule because I think that one, if it’s a default, people just don’t observe it. They break it way too much. That’s my opinion.

Marc: Well, I yeah, but I think you just gave a few exceptions that are good, right? Like I I’m not gonna necessarily schedule something ahead of time, but I will yeah, reach out to somebody and say, Hey, it would be quicker if we talked about this ad hoc. So maybe I mean, yeah, maybe it’s splitting hairs, but

Beah: Yeah, it’s interesting though, because I think of those as part of as like the rule is constrained. So I guess making a rule is is safer the more you constrain it. And you could you could arguably ha tackle one operational problem and either make a highly constrained rule that has exceptions and carve outs or a broader default and maybe sometimes both at the same time. So yeah, okay. All right, that’s a good one. give me you got another example.

Marc: Yeah, fair enough. Sure. Sure. Yeah, I I can give you another one. so this one we came up with recently and that’s working incrementally, which again, you would say and when I say work incrementally, I basically mean you know we’re we’re trying to ship smaller features more frequently or even larger features and smaller increments. you’re going back and you’re you know, making sure that you’re on the right track, etc. Which again sounds like obviously that’s something that you would want to do. But anyone who’s ever collaborated with anyone on anything, I think would would realize that, yeah, that’s I don’t think that’s a human default, right? Like I think we all, you know, want to have a task, go build whatever we’re supposed to build, and then show the awesome thing that we did. And yeah, that’s I think that’s a good instinct to have. But there are many cases where if you misunderstood something initially or new information pops up, yeah, it’s super important to make sure that you’re raising that up and collaborating with others on

Beah: Mm-hmm. So you s you said that we made that one recently. Like what does it mean to make an operational default? How do we codify or communicate these?

Marc: Yeah, that’s a great question. And I don’t think we’ve actually figured that out yet. We have a lot of defaults across the company. This one was was communicated to to several objectives. but yeah, it’s not I would say it’s written down, but it’s not there’s no like internal glossary or internal directory of of defaults or anything like that. It’s basically just a reminder for folks. And again, it seems like it’s something that you should obviously do, but I think it’s important to remind people of these things.

Beah: Yeah, so like we basically we write things down in the form of like messages and or or really I mean anyway, but and I think that can happen maybe one thing that’s worth saying is that that that can happen at different levels of the organization. Like sometimes you have a group of five people working on an objective and they might adopt their own defaults or ways of operating. And then on the other hand, once something once we like believe something to be

Marc: Sure.

Beah: Sorry, I don’t know if listeners can hear that, but my dog is drinking some water right now. So we’ll see if that comes through. you know, once we’re really confident in something and that it’s like broadly applicable across the organization, then we might communicate it at like a higher level and codify it a little bit more more strongly as a default.

Marc: Yeah, and that’s kind of where that’s kind of where we are with this right now. We’ve communicated it to a couple of objectives. You know, objectives end up having their own culture, intentionally or otherwise, and this is yeah, an attempt to kind of test it on a smaller care scale before we start communicating it more broadly. and I I have one other I can give. This one’s much more engineering centric, but so we’re all working on AI quite a bit these days.

Beah: Yeah. Okay. Yeah.

Marc: And I think anybody that’s ever worked with, well, anybody that’s ever used AI will tell you that or has observed that you don’t get the same answer twice, right? It’s it’s you know so-called non-deterministic output. so in order for us to build products leveraging AI, it’s really important that we have evaluations up front, right? That you start with, you know, say a list of if I have question why. or sorry, question X, I’m gonna get answer Y, or at least something in the general with the general gist of of answer y. and yeah, because you can’t always super accurately predict what’s gonna come out, it’s really important that before we start building something, you know, you you have this eval or evaluation data set so that you know when you’re doing a good job or or when you’re when you’re not. I think that one honestly that one’s fairly close to a rule in my book, in terms of the engineering expectations we have, but there are definitely cases where it, you know, it doesn’t make sense or it’s too much effort to start there.

Beah: Yeah. And so like because that’s a default, even if people most people kind of think it’s a best practice, codifying it as a default means that if somebody decides not to do it, they might get the question, why? Are you sure? And they have to like defend basically if you don’t use a default, you have to have a good answer.

Marc: Yeah, that’s exactly right. Yes, you have to have yeah, there’s there’s we encourage people to make exceptions, but it’s gotta be well reasoned.

Beah: Up there. Yeah. Yeah. Yeah, yeah. Cool. Can you think of any defaults that we’ve adopted as an organization or on a team and then like decided were the wrong default and changed them?

Marc: Yeah, I thought about this one a bit. I I can’t think of one where we actually decided this was the wrong default. But I can there there are several that we have you know, potentially evolved over time, and that continue to evolve. The one and and yeah, this one is also super engineering specific. But the one that that comes to mind is, you know, initially all of our back end engineers were the default for all of them was to use Perl for our back-end services, the Perl programming language. But we made exceptions there, you know, because no language is the right tool for every job. So, you know, if we were developing something that Perl just wasn’t the right tool for, we gave people the space to make a case to not use it. That and that was true for a number of years. I think now I wouldn’t call that a default anymore. I don’t think that that that is the starting place for most folks, like when we’re developing a new a new service. so maybe that’s one that as you would either say is evolved or or maybe retired is a better way to put it. But yeah, I I can’t think of one where I think we were actually outright wrong, but definitely new information and things come in and we change them.

Beah: Mm. Yeah. One that kinda one that comes to mind to me that maybe went from having a not really having defined a default intentionally, but let kind of the flow of the stream define itself was we we, you know, as some listeners may know, our company is organized into a a bunch of objectives, like teams working towards some important goal for the company. And those objectives are expected to hold what we call an assessment every so often to kind of like redefine the strategy in the roadmap. And we I think f for the first maybe f few years I was here, we didn’t have an explicit expectation on how often that happened, but I think the sort of adopted default was a pretty long period, like maybe on average, I don’t know, five months, six months. something like that and then at some point we what’s that?

Marc: Yeah, when you felt like it was important. I said when you felt like it was important, it was very I think very subjective.

Beah: Yeah. Well and it was like easy to procrastinate if you were leading an objective, you know, like ‘cause it felt like a time like a thing you’d have to put time into and put yourself out there. And so it just kind of was like getting long. But in reality we observed that when people ran assessments more often, we had better roadmaps and better decision making and, you know, sort of less leaving the track without, you know, the help to get back on the track and

Marc: Yeah.

Beah: And so at some point we I think maybe said like let’s default to every eight weeks and then we maybe t brought that back to six. Right now it’s it’s six weeks is the default. and I think that’s been a nice improvement to make that explicit.

Marc: It’s actually a it’s a good example of working incrementally over a you know, different horiz time horizon. But yeah, for sure.

Beah: Yeah. Cool. Anything else you want to add, Marc, before we wrap this?

Marc: I don’t think so. I think again, I feel like I have to keep saying that a lot of these are self evident because it just sounds so obvious in a lot of cases, but it is yeah, I think I think it leads to better results and and hopefully we’re gonna see that very soon in these objectives that where we’ve introduced it and yeah, r really important to challenge them as well.

Beah: Yeah, there’s probably a lot of obvious things that people don’t do on the regular because of some cognitive, you know, glitch or habit or, you know, your predictably irrational behavior. So yeah, just making it explicit I think can be powerful. Cool. All right. Well, thank you, Marc, and see you around.

Marc: Yeah, yeah. Yeah, thanks.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com.

Cristina: Hi, and welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering, or approach to AI. Hi, I’m Cristina. I’m on the marketing team, and I’m here today with Zack. Do you want to introduce yourself?

Zack: Yeah, my name is Zack Einhorn. I’m on the marketing team as well at DuckDuckGo and been here for nine years now. and I live in LA. Happy to be here.

Cristina: Happy to have you, thanks for joining. So today we’re gonna talk about our new ad campaign. On June 29th, we launched Prying Eyes, the name of our first national brand campaign for Duck.ai. And for those of you who don’t know what Duck.ai is, it is a way to chat privately with ChatGPT and other AIs, all in one place for free, no login. So today we’re gonna go behind the scenes, how it came together. Our unique development and production process, our views on AI, and more. Before we jump in, I just want to say to everyone listening: love it, hate it, have ideas for a new episode, please write us at podcast@duckduckgo.com. That’s podcast@duckduckgo.com. We will read all of them and would appreciate any feedback. So, first question: Zack, why why a brand campaign for Duck.ai and why now?

Zack: Yeah, well, you know, for years we had been focused on you know performance marketing for as an alternative to Google and Chrome and focusing on search and browse. And we’re introducing a new product market that has virtually zero awareness. And we’re really confident about the product market fit. We know that there’s a lot of built-up anxiety around AI and a desire for private alternatives. And so we know it’s time to let people know that one exists and from a company that you trust and have been trusting.

Cristina: So I find that when I chat with people about what I do, they think the the making the ad portion of my work is the most interesting, right? And they’ll ask, like what is it like to be on set and all these kind of things? So I think a lot of people find that industry but interesting. But even for industry veterans, it’s maybe interesting to hear about how we at DuckDuckGo make an ad, because it’s a different bit different than how most companies do it. Can you dive into that?

Zack: Yeah, absolutely. so you know, most companies, and in my in my experience, the prior companies, you know, they kind of hand everything over to one agency. You know, everything is sort of with one big house strategy, research, testing. and the way that we do it is a little bit different. Like you said, we’re we tend to we like to be really hands on. and I think, you know, one thing that makes us pretty different too is that, you know, we’re we you know we’re some of like the foremost researchers on this space at the company. And so we you know, whereas it might be really useful if you’re in like a you know a a a category, like you know, consumer package goods, to have like the expertise of an agency that may have many, many clients in that space and can and can, you know, give you strategy and insight into areas that you may not have. but because we really you know, we have a lot of that research and talent in house, the strategy really you know comes from us. and you know, we tend to lean on agencies more for like additional initial ideation. and then from there we really take things into our own hands. We have we have freelancers that we’ve gone back to a handful of times to help us develop ideas. We’ve got internal talent that helps us develop ideas. and then we have production companies that we work directly with. And that I think is a real for us. You know, we we save a lot of, I think, you know, we save a lot of time in the game of telephone that you usually play with your with your agency by being able to, you know, handpick the director, be you know deeply involved in the post-production, like attending edit sessions, coloring. and I think that that gives us a advantage because you know not only do we get to accelerate by making those being hands-on in those choices, but we also continuously build those skill sets. And so you know we’ve been doing it now, we’ve been doing it this way and kind of building on that process for five years now. And we’ve just we’ve built up a a a ton of expertise internally and done a lot of hot like you know, a lot of good hiring to fill the the areas of expertise that we didn’t have, expanded our network of partners. yeah, we’ve really kind of built up this this in-house team.

Cristina: Yeah, thanks for walking us through that. So internally, the team has been calling this ad Prying Eyes. Like, what does that mean? Where did the idea come from?

Zack: Yeah, absolutely. So it came from one of our one of our agency partners. You know, we we’d we went through a process where you know we realized one of the things that we weren’t super strong at internally was coming up with the initial idea. and so we ran through a process where we worked with a series of agencies and freelancers to to come up with ideas. One of them came up with this idea that was really sticking with us where we had you know giant photorealistic eyeballs following you know c following people and and you know in you know invading their space when they’re doing asking AI sensitive questions. And we had a few directions we were looking at, and we had done a bunch of research. And the thing that we kept coming back to was this thing we called exposure anxiety. And the crux of it was that we realized that when we got a level down into the specific things that people were worried about being spied on. you know, giving away when they’re when they’re using AI, that it became like a bit polarizing. Like, you know, they once you used a specific example, people are like, well that’s not quite really me. But we found the thing that really stuck for everyone, and that was the most visceral thing, which I think is the most important because I think that’s what sticks with you it emotionally in an ad and will make you think of it later and motivate you, was just the actual sensation of of being watched. That was the through line that whether you were worried about, you know, asking AI about something related to, you know, health, or whether you were asking about an issue going on at work, or you know, whether you’re asking AI you know for symptoms of, you know, something that your that your kids dealing with, that that no matter what it was, everyone shared this this this feeling of exposure anxiety and you know what what can make you you know what is more exposing than feeling like there are giant eyes behind you watching everything that you that you type.

Cristina: Yeah, this was this was a fun concept. you know, we considered a lot of different ideas, and for me, just seeing and thinking about the eyeballs was just so arresting, right? In a really clear visual way to encompass the feelings that we had heard in research, right? We’re a very data-driven company, and so research is really important to us. We’re not just thinking of something that looks cool, it needs to be grounded in an insight that we think relates back to the problem we’re trying to solve.

Zack: Yeah, absolutely. And I think it also just there was a lot of connectivity to to like our overall message, too, like, you know, in in terms of of privacy and and and prior campaigns. I think that we, when we went in, I think we really thought we want something that is uniquely AI. but I think what we found is that a lot of the anxiety around AI is kind of an evolution of the of the anxiety that already existed. So just like the way, just the way that AI is the newest way that people are are using the internet and and how it’s you know it’s more personalized and you know tailored than than than search and browse was it’s also that much more viscerally you know uncomfortable you know you’re you’re a lot more you know, likely to have sort of these uncanny experiences where, you know, AI will bring up a prior conversation or mention a tidbit that you thought, you know, wasn’t being logged, you know. and and so I think it really helped draw that draw that through line.

Cristina: Yeah. Yeah, I I personally feel very passionate that if someone is going to use AI, they should use Duck.ai because it’s hard to anticipate just how this data will be used from other companies in the future. But I think we can all anticipate, probably not in a great way, right? In terms of privacy. On the last episode I was on, we talked about how not everybody wants AI, right? And that’s why we feel so strongly about making it optional. So how do you think about making an ad for AI?

Zack: Yeah, I mean I think that’s a very real subset of people. I look at, you know, the way that I look at DuckDuckGo’s mission is, you know, I think we, you know, I look at the goal of DuckDuckGo to provide for people the the best quality possible version of the internet that you are used to using, but private. And so I think, you know, that is the that is the the the mass of people that we saw that that we that we serve. AI adoption is, you know, well over fifty percent and growing. And so I think you know it’s really important that we serve that audience. at the same time, you know, we firmly, while while we believe, you know, in providing the pro in private AI, we also, you know, we also take exception to the way that AI is being sort of jammed down people’s throat by other by other companies. and you know, we take pride and we think it, I think I we think it’s you know goes along with our overall mission of build of raising the standard of trust online. to give people the ability to control AI and use it on their own on their own terms. And so even in our in even in our our advertising of our AI products, we’re very clear you know in in our messaging, like for example, the television ad in our end card and VO to say that you know Duck.ai is is from DuckDuckGo and that our our policy here is that AI is private and optional. And so we you know we want to be clear that that is all you know that is our brand’s stance.

Cristina: Yeah, absolutely. So we’re spending all this money on TV and digital and all these channels. How do we know if it works? Like what what are we trying to achieve here and what are those signals?

Zack: Yeah, I mean it is, you know, raising brand awareness is meaningfully harder to read than than you know performance performance marketing. and it takes time. you know, we we I think the bottom line is we won’t know, we won’t know right away. the what we’ve seen from the initial tests is you know, we we’re able to get a read on awareness on a two to three month kind of timeline to see if we’re moving that. usage usually lags behind that. and so I think, you know, a lot of times when you’re when you’re looking at awareness, I think what you’re looking at is is, you know, you have you have immediate you know you have immediate key performance indicators which for us are all are all centered around awareness you know we’re doing pulse checks for every four weeks we’re we’re revisiting a survey and we’re seeing if we’ve if we’ve raised it. and then in terms of driving actual growth, I think it’s really more of a higher level question of running that strategy for long enough to be able to create the the flywheel and seeing how it impacts growth holistically. And so I think that is something that’s probably on a longer you know, timeline closer to, you know, six months to a year where you can really see the the big impact of focusing on on brand awareness. And that’s that is similar to what we saw you know, several years ago when we were doing big brand awareness push for the core product.

Cristina: Yeah, absolutely. It’s a bit of a patience game, right? Wait waiting to see if these metrics go up and maybe leaning on some early indicators, but it’s not that instant gratification that you get from a direct response camp.

Zack: Yeah. Yeah. Yeah. We are looking at the direct response metrics on but but trying to set expectations that that you know we’re in a we’re in a teach people our name place.

Cristina: Yeah. So so you and the team have been working hard on this for months. Maybe you get to take a little bit of a break now while, you know, the big channels are out there and you’re waiting for some of the data to come in, but I’m sure that’s not all. So do you want to speak to what’s next?

Zack: Yeah, I mean I think you know, not really taking a break. trying to, you know, I think we had real we have really you know, ambitious goals for this campaign. I think, you know, we we’re we are ramping up in all senses of the word. You know, we’re starting out with we’re starting out heavily in a in a in a handful of channels, very, very video centric, you know, CTV, YouTube, and and social, but we’re really looking to build out the 360 of the campaign. We’re looking at a handful of other channels inclusive of out of home and audio and looking to expand overall coverage pr we’ve got we’ve got a bunch of a bunch of stuff deck to to help fill out this campaign and and bring it to life on a larger scale.

Cristina: Yeah, absolutely. I’m very excited that this is finally out there now and working to not only teach people the name, but hopefully get people using it. So again, if you haven’t yet, try Duck.ai. Again, it’s free, no login required. So if you want to do an AI search, but you want to do more privately, give it a go. Zack, it was really great chatting with you today and kind of reliving the work of the last few months. Thank you and the team for all of your hard work. and again, very exciting that the ads are out there now.

Zack: Mm-hmm. Thank you. Appreciate you having me. I’m very excited.

Cristina: Alrighty. Yeah. Okay, well that’s all for now and stay tuned for future episodes of Duck Tales. See you later.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com.

Show notes: Ad Blocking is enabled by default for Mac, iOS and Windows browsers. It will be enabled by default for Android soon, but for now can be turned on in settings.

Peter: Hello everyone and welcome to Duck Tales, where we go behind the scenes at DuckDuckGo, we discuss the stories, technology, and the people that help build privacy tools for everyone. In each of these episodes, you’ll hear from employees about our vision, product updates, engineering, or things like approaches to AI. My name is Peter. I’m on the product team at DuckDuckGo, and I’m joined with my colleague Dave. Dave, do you want to introduce yourself?

Dave: Sure. Yeah. Hey, I’m Dave. I generally work on the browser experience at DuckDuckGo. That means working on privacy protections, working on making sure that websites work the way that you expect them to work and yeah, making sure that you have a good experience browsing the web.

Peter: Awesome. And today we’re gonna be talking about ad blocking, more specifically YouTube ad blocking, but I think we’re gonna talk also just generally about about ad blocking. So Dave, why don’t we why don’t we start at the top? Does DuckDuckGo browser block ads today?

Dave: If if you’ve used one of our browsers, you’ve probably noticed that you see quite a few fewer ads. So that’s a side effect of our tracker blocking. So the reason that you’re seeing fewer ads is a lot of these ads come with trackers, you know. They want to see how you interact with the ad. They want more info about you to determine, you know, is that like a a good impression? And you know, they want to build these profiles and all of that. So we block those requests before they’re initiated. And that leads to many fewer ads across the web. But we don’t deploy a like a specific ad blocking feature.

Peter: Mm-hmm.

Peter: Got it. And so we what what you’re basically saying is a lot of these companies that, you know, serve up advertising are also tracking the heck out of people. And so by by virtue of the fact that we block that tracking, we are basically blocking you know a lot of the advertisements on the internet.

Dave: That’s right. That’s right.

Peter: Makes sense. And so what’s changing about our ad blocking?

Dave: Yeah, so what we’re doing here, and as Peter said, we we’re starting with YouTube. We r have received a ton of user feedback asking for us to do something about this. As as you probably know, YouTube shows up to you know several minutes of ads before videos, during videos, just constantly. And what what’s changing here is we have deployed we’re working on deploying a feature that will just prevent these ads from loading. So instead of, you know, waiting thirty to seconds to a minute before you can watch a video and then getting interrupted in the middle of a video, you should be able to watch the video right away and without interruptions.

Peter: Yeah, and so a a lot of you’re saying a lot of our users have been asking for the ability to block ads on on YouTube. I think it’s safe to say that, you know, a lot of DuckDuckGo search users who historically have used other browsers, have have used extensions, ad block extensions to get this benefit. And so Dave, you’re saying a lot of these users have asked us for the similar sort of capability in our browsers.

Dave: Yeah, that’s right. Extensions are a very powerful tool. We don’t export extensions in our browsers for for good reason. So they can be a blessing and a curse. If if you’re using the right extensions, there are safe extensions and they can improve your browsing experience by like you know blocking ads or you know auto-filling password fields, that kind of stuff. But they’re also a a really large risk because there are a lot of malicious extensions out there and unscrupulous extensions out there that can you know get full access to everything within the web pages that you’re looking at and exfiltrate your data and track what you’re doing online without you even realizing.

Peter: Mm-hmm.

Peter: Yeah, I I remember over the last few years, every few months there seems to be a story about some extension that that a lot of people installed that either went rogue or maybe got sold to another company and then they use that extension to to data mine people and spy on their their online browsing behavior overall. And so you’re you’re basically saying, Dave, by adding this capability, the YouTube ad blocking natively to our browser applications. Not only is it easy for our users to enable and get that benefit, but it is also safe to do so. Is that right?

Dave: That’s exactly right.

Peter: Awesome. Cool. And so we built this based on user demand. But a lot of our browser users already have the benefit of something we call Duck Player. Can you explain what Duck Player is and then let’s talk about how Duck Player differs from the YouTube ad blocking that we’re talking about?

Dave: Yeah, totally. So Duck Player, we like to think about it as kind of a theater mode. So if you visit YouTube or you load a YouTube link, you can load it in this kind of standalone window within our browsers that we call Duck Player. And what what’s happening there is we’re loading it through this separate YouTube no cookie domain. And and the advantage of what we’re doing there is that it doesn’t like when you watch videos, they’re they’re not tied to your account and it they won’t influence your viewing history or your, you know, recommended videos. But over time we’ve we’ve seen quite a bit of feedback from users about issues that they’ve run into with Duck Player and that that have made them go back and watch videos on YouTube’s main page. So I can go into this a a little bit here. So quite often, I guess all the time, when when a when a content creator uploads a video to YouTube, they can decide whether they want that video to be viewable on YouTube or whether they want it to be able to be embedded on other sites. So if they say, no, we don’t want this to be embedded on other sites, that makes it so it also can’t be watched in Duck Player. And there are a couple other edge cases here that prevent users from watching videos in in Duck Player. So if there’s an age restriction on a video that you have to sign in to your YouTube account in order to watch the video, that means you also can’t watch it there. And you know th they’re there a variety of issues that we’ve run into or heard from heard about from users.

Peter: That makes sense. Yeah, I I think I think it’s safe to say that our our users love Duck Player. A. We we’ve heard from a lot of users that it’s it’s a killer feature that they they use all the time. But as as you said, yeah, between people who do want sometimes they want to see the full full comments and things like that that you’re not gonna get in Duck Player, they want to see those on YouTube, but they also want to benefit from the the ad free experience. That’s part of the reason we built the YouTube ad blocking and to to avoid some of the error messages that you referred to, which tend to come up pretty often, particularly on on desktop browsers, the error where video cannot be watched because a content creator disallowed it from being watched outside of YouTube. It comes up a couple percent of the time overall. So with this new YouTube ad blocking, it should be a little bit more seamless for everybody. Cool.

Dave: Yeah, and I I just want to make it clear that Duck Player’s not going anywhere. That that still exists. You can still choose to use it. We’re just creating more optionality here.

Peter: Awesome. And and that means that anyone who’s currently setting Duck Player to always load videos in Duck Player, they should continue to have that experience as their default, right?

Dave: That’s right.

Peter: That’s great. Talk talk a little bit about building out the YouTube ad blocking capabilities. What what were any main challenges we hit along the way?

Dave: Yeah, for sure. Yeah, so as you know, we have a we have a browser on four different platforms, Android, iOS, Mac OS, and Windows. Slightly different architecture on each platform. Websites literally differently between, you know, they they often have like a mobile version and a desktop version. So it can be a challenge to ship parity across all all platforms. I think the the trickiest part about building ad blocking for YouTube is it’s it’s not just a simple block this request and ads won’t load. I think you know YouTube and other large websites are incentivized to prevent users from ad from blocking their ads, which means they make it a a little bit more challenging to work around that. So what we’re doing here is, you know, we we are blocking a few requests, but we’re also injecting several different scripts into the page that kind of you know rearrange how the ad calls load into the videos.

Peter: Hmm. And and if if you our users who are using the new capability, if they if they see issues come up through their use of it, is there a way that they can tell us about that?

Dave: Yeah, that’s a good point. Absolutely. Please, if you run into any issues using this new feature, you can either submit browser feedback, which is through through the settings page. We also offer a more direct way to submit feedback about a problem you’re experiencing with a specific site. We call it a a broken site report. You can access that through the privacy dashboard, which can be found by clicking the the little shield icon in the address bar. And if you just fill that out, our team does read these reports and we we try to we we do everything we can to respond to them as quickly as possible.

Peter: Awesome. I think it’s also fair to say that our team at DuckDuckGo is pretty active on Reddit, particularly the DuckDuckGo subreddit. And so if you prefer, if you want to post something there about your experience, by all means, we we will we will engage with you. We’d love to hear you know the direct feedback and and and sometimes help troubleshoot. And that that goes both ways. We certainly like to help people if they run into any issues, and that helps us understand if there are some issues on particular machines that we may not be seeing internally and helps us chase those issues down overall. Awesome. Dave, what’s t tell us a little bit about when people can expect to see this.

Dave: Yeah, so as we record this, it’s currently rolling out to each of our platforms. By the time that you are watching or listening to this, you should be able to access it on all four of our platforms. If you you know, we’re we’re enabling it by default for users, and you can also go into settings and check there’s a new there should be a new ad blocking menu item there, and you can, you know, make sure that it’s enabled or disable it if you don’t want it, if you like ads.

Peter: Cool. Well I hope all of our users are ex as excited about this feature as we are, those of us the internally that built it out. And like we said, we’re always eager to hear feedback if you have any. And with that, Dave, thank you for joining me on this Duck Tales episode. See y’all next time.

Dave: Thanks, Peter.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com

Beah: Hello, and welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, the technology, and the people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering, or approach to AI. Today you’re going to hear about our approach to AI in the browser at DuckDuckGo. So we have—we have four browsers. Or we have a browser for four platforms, Android, iOS, Windows, and Mac. And I have with me to talk about this topic today about how we have integrated AI into those browsers, Aitor. Aitor, would you like to introduce yourself briefly?

Aitor: Sure. Yeah, so my name is Aitor. I’m originally Spanish, currently living in Portugal. Been in DuckDuckGo for, I don’t know, five years, something like that. I don’t remember anymore. Started as an—as an Android engineer and now I’m leading with other folks what we call the objective for our AI browser integrations, which is what Beah just presented.

Beah: Yep. Awesome. And yes, if—if we—if you haven’t met me before, I’m Beah, I’m on the product team at DuckDuckGo. Before we jump in, I have a bunch of questions for Aitor. But before we jump into the questions, I just want to say this is a PSA to the audience. We’ve been running Duck Tales for, I think, a bit over six months now, and have a pretty good following on Substack, and we’d love to hear from the listeners more.

Aitor: Okay.

Beah: So if you have feedback about what you like, what you don’t like, what topics you’d like to hear more about, send us a note at podcast@duckduckgo.com with your feedback. We’ll also put that address in the show notes. But please, we’d really love to hear from you. So okay, with that said, maybe just a quick kind of introduction to how we’re approaching AI in general at DuckDuckGo before we get into the—the details of how we put it into the—the browser. Since we started adding AI features to our products, we’ve had this like three-prong value system that AI be private, useful, and optional. So I think those are like—private’s probably pretty self-explanatory if you’re listening to this podcast anyway. Useful meaning like not AI for AI’s sake, but actually like solving a meaningful problem for users. And then optional is also maybe deserves a little explanation. Like you can turn AI off in all of our products. So you know there’s an incredible spectrum of opinions about AI and when to use it and how much to use it that ranges from never to all the time and everywhere in between. And we put a lot of care into giving users the like toggles and controls they need to customize their level of interaction with AI. Whatever you choose there, it will always be private. So that’s like a—just a little bit of groundwork. But Aitor, can you tell me a little bit about like how that—how we’ve integrated AI into the browsers and maybe a demo would be in order too.

Aitor: Yeah, let’s just do—let’s just try to share—this is where everything goes wrong now. Let’s just try to say how our AI integration looks like in production right now. Let me share my phone. Okay, so can you—can you see that?

Beah: Sure. It can.

Aitor: So this is how people would basically see our browser whenever they—they open it. In here you have like some favorites. It’s just for—for display. So the way that we’ve integrated AI in the browser right now is through this toggle, as we call it internally. You see above—search and Duck.ai. What we want is to give like a very quick access to search, a very quick access to Duck.ai, and—and a very easy switch between them. So in here, users could start a search or a browse. Similarly they can just switch the toggle or flip the toggle to the other side and they would see basically our integration with chats. You see here there is a bunch of chats, different icons to show whether it’s a voice chat or a text chat. We also want to have this marriage between chats and—and browsing and we’ll get—probably a little bit more into details later. So at the bottom, if you didn’t want to chat, you have like an easy access again to browser—to search. You can access one chat right from here. Then basically we display our Duck.ai chat. And from here you can just continue chatting from here. This is in Spanish, sorry, or you can just access—or your history or you can create a new one. So this is basically how—

Beah: That looks like a testing history if ever I’ve seen one.

Aitor: Yeah. So this is basically how—how it looks like in production. It will change very, very soon. So we are gonna ship like a—this is like a good experience, but we are gonna improve it like even further. And yeah, in here related to privacy, like you can just like delete your chats for good, you can rename it and this is like our fire button, which if you are a user you should know about it. The context of—of a chat, you can just basically delete a chat and discuss with our privacy story as well. So this is like a very—

Beah: Yeah, I love that. I love the chat level delete. Like I—like I don’t—I don’t use the app level fire button hardly ever ‘cause like I want some amount of memory, but I have like a bunch of like quick throwaway chats like all the time and it’s so satisfying to get them out of my like history list, yeah. Yeah. Hey, you mind going back to the toggle—sorry, Aitor.

Aitor: Yeah. Yeah. Me too. Yeah, I’m also like a heavy user of this one. Yeah, cool. No, no, I—I was saying that I’m—I’m kinda like the same user. Like I don’t generally use the button to just clear everything. I’m more surgical. So I love the fact that—that this is the integration and experience that we have—that we’ve created for chats.

Beah: Yeah. Do you mind going back to the toggle view? I just wanted to like the search or Duck.ai input. So like when you were—yeah, if you go to the Duck.ai side, just to be totally clear, like those are historical chats. So we—so the auto complete is filling—like if you wanted to revisit a chat and if you start typing it’ll filter. So if you have like a very long history list, it’s an easy way to get to—yeah.

Aitor: Yes. Exactly. Yeah. Exactly. Yeah. This is the chat history—similar to—it resembles basically the experience of search, although it’s like slightly different. It’s only about chats. And—and yes, as soon as you start typing, it starts autocompleting the ones that they are more relevant to—to your—to your—to your text. There’s a bunch of—of things that we are doing around improving the—the—the searching capabilities. So that it’s not only by title, but it’s like—I don’t know, if you don’t quite remember the exact words of your chat, basically we still wanna present with that option, right? So there’s like a bunch of work as well going on in that direction.

Beah: Yeah, it’s interesting. Like it—well, I think one of the s—central product challenges here is like chat is not entirely analogous to search or browse in that you—you know, you pro—you don’t want to like continue a search. I mean you might like branch off from a search, but like what—you know, an old search is an old search. A chat you might want to like revisit a chat to continue. It’s like a living thing that you wanna continue. And so that’s created some interesting, like—we’re—we’re sort of trying to make these parallel worlds that will feel familiar to some extent.

Aitor: That’s fine. Exactly.

Beah: Good thing. I can’t read Portuguese, huh? Or Spanish. Okay, no. I saw the heart though, that was nice. Yeah. We can edit that out and post if you want. Very good. Good, ‘cause I like it. Yeah. Cool. Do you mind—Aitor, there’s like another—do you want to just show the sort of—if you’re on a website, like how to—how to start a search or a chat?

Aitor: I know, that’s totally fine. Set, live demo, if things happen. Yeah, so if we go to like a—a website, in here what—what the users can do to quickly go into—into chat is there is this little CTA, this little icon here where users just can tap it and this pops up this sheet that allows the user to chat with the context of—chat about the context of—of the page that they are browsing if they want to. This is what this means basically here. This means that the context of this particular page is attached. You see like BBC Home, that card in there. User can remove it if they just want to chat about something generic that doesn’t really have to do with—with a page. And you can reattach it if you want. And then we have like quick accesses here that says summarize this page. So if the user clicks on it and then submits, you should just get like a summary of—of the page that you were browsing. So this is the Gombia.

Beah: Thanks. No, I just said nice. Go ahead.

Aitor: So this is the quickest way that you can get into—into chat once you are like in search or browsing. From here you can do different things. You can continue chatting about—about the context of the page or any—or anything else really. You can expand this, this—this button basically. So if you wanna bring this into kind of like the full experience that you saw before, so you can just go there. And your chat is of course always like stored privately in your phone. Like this doesn’t go to any cloud or anything like that, like the history and the chatting like is—is fully on device basically. So that’s—that’s basically the—the fastest way to interact with chat when you are in—in a browser search mode.

Beah: Next. Cool. So looping back a little bit to—I was talking about optional being one of our core tenets of AI, how—how do we give users choice in these features?

Aitor: Yes, so every—every pretty much like everything that you’ve seen, and this is one of the challenges, has an associated setting. A lot of it has. So if you—if your user goes to AI features, this is where anything related to AI in the browser lives. And here you can find several things like Duck.ai—this toggle basically controls whether you will see Duck.ai at all. In our browser, as those like buttons, CTAs—CTA is like the quick CTAs like a button, if you will. So those buttons, those entry points, this is the master toggle for that. Then if you have it enabled, we provide couple experiences. The experience that you’re seeing is the toggle, which is this one in here, which is the one that we believe makes the experience better. But we give optionality to users if they just don’t like that toggle and they just like go to the regular address bar. We can—we can demo that a little bit later. And then in regards to whether or not we are gonna auto-attach the context of the page that you’re browsing, then you have this toggle here, this setting here that basically says automatically send it, which basically means as soon as you open that bottom sheet, we are gonna auto-attach the context so that you don’t need to do it if that is the most common action that you take. Or you can just disable, in which case like when you open the sheet, the context will not be attached and you have the optionality to attach it by—by that button basically that you saw in there. So if I disable the toggle and then we go back and then we create a new tab. You see that now you just don’t have that toggle, which is that nice affordance to switch seamlessly between search and Duck.ai, but still you can get to Duck.ai via the chat icon that we have in there close to the—to the mic. And then if you click in there, you just have access to like your new chat and then it’s exactly the same experience. Same goes for if you are browsing. So if you go to one page, you don’t have the toggle, but it’s still—you still have the same access to the sheet so that you can quickly chat with or without the context of the page that you are basically browsing at that point.

Beah: Yeah, the fact that I have also—Yeah, that’s—I’m thinking about—that’s because that—was—Got it. Thanks. Maybe like switching gears a little bit, you might—I don’t think you need to share your screen for this one, but if you wanted to close it down. Tell me, so like, h—integrating AI into a browser—or a browser that has AI capabilities is like a pretty undefined space compared to many things that we’ve worked on. Like it—nobody’s really figured out exactly how this should work and the capabilities and user goals are like changing every month. So I’m just curious, given all of the ambiguity around it, are there things that, you know, we’ve tried and haven’t worked or think like major learnings from the last year or so of working on this?

Aitor: So a good question. It’s hard to say. Like major learnings—well, like in regards to like the context of integrating AI in our browsers, like this is kinda like—it’s very hard in the sense that you cannot just—not the type of problem that is well known. You have a bunch of like other people doing the same thing and you can just like take inspiration for what is out there. Like I think pretty much everyone that is trying to do a similar thing is in the same position. So it’s very hard to understand what the users want and it’s very hard to anchor the experience around the right use case, which is in reality like—with AI, you can do anything. You can put AI on anything in a browser. You can do anything with AI. The hard thing is to understand what is actually valuable to the user versus what is like a more gimmicky type of stuff that users will—are gonna use it once and never again. So that’s I think is the hardest—one of the hardest things and challenges that—that—that we see. What—what is the—what have I found? Like I think when I came into—like this is like a personal learning to be honest. Like I don’t know if I—it applies to everybody, but when I came into the objective or integrating AI on the browsers, like I was more on the—on the spectrum of let’s just try to provide features that resonate and—and—and find that use case, anchor everything around that use case, right? And I think we—we initially realized very fast—realized that we—before we enter on that space, we needed to provide the right mechanisms for people to just access chat in a seamless way. So like we were always talking about from—open up to a search, open up to a chat, open up to continue your chat, continue your search, right? So I think that was like—for me personally, was kind of like a—I had to switch my—my—my mental model from let’s just provide value directly into the AI versus what we have is good. Let’s just give like the right accessibility, visibility and—and the right access points to it that—that it becomes seamless. That was probably for me personally the first learning that I got into getting into this objective.

Beah: Mm-hmm.

Beah: Yeah, like if you build a feature, they will not necessarily come unless you make the access to it and the discovery of it really, really good. And that’s a hard thing to do, I think, in browsers where we don’t—people don’t really have like super well-formed mental models, and then especially in a browser that’s like this big, like the size of a phone, you know, when you only have so much surface area to help a user find what they want.

Aitor: Exactly. Yeah. Exactly. Plus it’s very easy with a—at least for me, like it’s very easy to get excited with AI. Like there is like—like I said, like you can throw AI at anything basically and anything looks good and—and seems useful. So it’s very easy to get like—happy run about like—just just add all a bunch of like cool features and interesting features that you don’t really know whether they are gonna resonate or not with the user, but then you might miss like a little bit the point about let’s just make it very easy to get access, like very easy to use. And once you anchor that, then you can just move on to—to the—to the use cases that resonate more with the—with the users. And this is also like some of the learnings that we’ve seen in some of the user research that we’ve seen, which is—it’s not that some people—it’s not that they just don’t like AI, like the asset—like there is like you said, like there is like a whole spectrum from people that don’t like it to people that love it, and then anything in between.

Beah: Yeah.

Aitor: If we take anything in between. Like a lot of people they know it’s there, they wanna use it, but they don’t quite find the context where it’s useful. So this is one of the things that we’ve seen with people that—okay, I’m here, I wanna use it, but I just don’t know how—how would I use it now or my context or what I’m doing on my day to day, right? So I think this is one of the hardest part.

Beah: Yeah. Maybe that’s a good segue. This is my last question. Is there like—what’s your favorite thing about how AI is integrated into our browsers? And for instance, do you have like a use case, you know, a—how to use it that’s been very satisfying to you that others might appreciate too?

Aitor: So for—my almost like my number one use case is use context. So whenever I browse about something, I just go and say summarize this thing for me and then start chatting about it. I think that is probably one of my top usages. Yeah.

Beah: So you like go to a web page and you add the page context into the chat and then you can talk to the chat about the page context. Yeah, yeah.

Aitor: Exactly. Exactly. Yeah. That’s—I think that is the one that I—that I use the most for different things, like to compare things on a website to—to just because I’m lazy and there is like a bunch of like news and I want a TLDR before I decide what I click through. So there is like a bunch of like use cases for me. I also like—I use the same flow to be honest on a day to day to just start like a new chat. So if—regardless of what I’m doing, I open the browser and then I open the sheet and then I just don’t attach the context and I go to a chat immediately and start chatting about—I don’t know, the last exercise that I did and how—how I perform or not perform and things like that. Like anything that you would do with your—with your AI, right? But those are the two—the two main use cases for me that I pretty much do like every other day. And as far as accessibility goes, I personally like the toggle to be enabled—just just give me a much easier access between our browse mode and our chat mode. So I’m—I’m a toggle user—an a—a toggle user specifically.

Beah: Yeah. Yeah, same. Sometimes I like—don’t know whether like—I start typing and I don’t know yet whether I want search or chat. Like there’s—I have a—for me, there’s a bunch of like prompts or questions that I might ask that like are in a shared space, like just depending on what I feel like or my prediction for what’s gonna come back. I mean, especially with search assist and our search results, like sometimes I’ll start chatting thinking I want a search assist answer. And then I realize like my question’s getting long enough that probably what I really want is a, you know, chat and I can mid—mid-typing like swipe the toggle and you know it’s pretty satisfying. Cool. Awesome. All right. Well thank you so much, Aitor. I appreciate you joining the pod. And yeah. All right, talk to you later. Bye.

Aitor: Yeah. Yeah, exactly. Yeah. Thank you, Beah. Thank you for having me. Yeah, thank you. Bye.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com

Cristina: Hi, and welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering, or approach to AI. We’ve been running Duck Tales for over six months now and have a strong and engaged following on Substack, especially. We’d love to get your feedback. What do you like or dislike about Duck Tales? What topics would you like to hear more about? Is there anything you’d like us to go deeper on? You can write to us at podcast at duckduckgo.com. That’s podcast at duckduckgo.com with suggestions. We will read all of them. And you can find our email in the show notes too. So with that said, hi, I’m Cristina. I’m on the marketing team. And today I’m here with Kamyl, who leads our communications and public affairs teams. And we’re gonna interview each other today. Kamyl, do you wanna say hi or anything else?

Kamyl: Ha ha. Greetings everyone. Thanks for tuning in. Cristina, I’m really excited to talk about our topic today with you.

Cristina: Awesome. Okay, so to jump in, DuckDuckGo has always been known for privacy, but lately we’ve been talking a lot about AI choice. How did private, useful, and optional become the lens we put on AI?

Kamyl: So I remember when ChatGPT first launched and I was talking to Gabriel a little bit about, you know, how do we think this is going to show up in DuckDuckGo? And I think one of the things that makes working here at DuckDuckGo really fun and different is that we get the freedom to think a little differently, and to say if everyone is doing one thing, how can we do it a little different. How can we think about AI in a way that’s uniquely DuckDuckGo? And that’s how we landed on private, useful, and optional. Private, what most people know about DuckDuckGo, we’ve always respected our users’ privacy, never collect search histories, anything like that. And we do the same with AI. Useful in that we never wanted to just launch an AI feature. Because it’s AI, but it actually had to do something helpful. It had to do something to improve the search experience or the browsing experience. And then optional, knowing that there are many people out there with many different opinions about AI, and opinions are still forming. Some people want the ability to use it to the their most its most extent. And some people don’t want to see it at all. And that, you know, because of a lot of big tech investments in a in a in AI like Google, they aren’t willing to necessarily just give AI a straight up off button. So we knew that those were things that we could do uniquely, put them together and then that became our lens.

Cristina: So because we are in this optional standpoint, we offer a no AI experience, we also offer things like Search Assist and Duck.ai. How do you respond to folks who might say that offering this range is contradictory?

Kamyl: Right. You know, I think it’s part of the it’s a major part of our goal at DuckDuckGo to let people use the internet how they wanna do it, but more privately and without trade-offs. And so some people want to use the internet with AI and some people want to use it without AI. And so to us, giving people a way to do whatever they want in a more private way without trade-offs feels really core to our mission. And so it’s not contradictory. We don’t have to have sort of one binary stance on it. We have to think about who our users are and what they want. And instead of forcing them into one experience, we think it’s the right thing to do to let them choose.

Cristina: Got it. So for you and I and the team, it’s been a very busy, very exciting few weeks. Yes. And this all started at the Google I/O conference. Can you tell us what was that and why did it have an impact on us?

Kamyl: Indeed it has. Yes. So Google I/O happens every year. It’s their developers conference and they announce you know, a extremely long laundry list of new features and products and things that they are excited about. And at the top of that list were changes to core Google search. Essentially that they were going to turn it and are in the process of turning it into a more AI forward experience that is turning the search bar into an AI chat bar. And essentially the moment that that happened we saw organic conversations about it explode over the internet. Why is Google doing this? They’re getting rid of blue links. Here we go again. The issues they we saw on AI Overviews. And it was a moment where you know, if Google has been trying to kind of boil the frog in terms of really slowly changing the experience to turn it into pretty much a full AI chat experience, this was the moment that the frog started to jump out a little bit. And that’s sort of what we saw on social, that people sort of saying. We have enough of this. This is making it worse. This is the last straw. And as people started doing that, you know, we saw people recommending DuckDuckGo in really high numbers. And also starting to download DuckDuckGo in much higher numbers than we were seeing before. Over the two weeks or th— really three weeks since the Google I/O conference, our installs are up thirty to forty percent above baseline. And so it was really exciting to see DuckDuckGo being a part of this conversation and people recognizing after you know talking about optionality for a long time, sort of people recognizing that that’s a really important part of our product and coming to us for that reason.

Cristina: Yeah, absolutely. I mean, we saw this amazing conversation a folding unfolding around us. Can you go a little deeper in terms of how we joined the conversations, worked to reach more people and some highlights of what they were saying?

Kamyl: Absolutely. There were multiple Reddit threads about, you know, not only the the the news that Google was making these changes, but then about how much we had grown. At one point we had announced that search that that installs were up almost eighty percent, almost seventy-five percent from a pre-Google I/O baseline. And once this news got out and people were reacting to it, it it really just exploded. It was kind of an important social proof, I think, that people were interested in seeing that, you know, people aren’t just complaining about Google, but they’re actually trying out different things. We’ve even done some research about, you know, who are these new folks coming to DuckDuckGo and what are the reasons that they’re that they’re coming to us. And one of the top reasons is that they’re unsatisfied with Google’s search results. That the quality had gone down to to an extent that they had sort of finally had it. And, you know, of course many users feel trapped within the Google ecosystem, so it’s not exactly easy to leave. But I think installing us was sort of the easiest way to do that. And you know, seeing so many threads on Reddit, for example, with twenty K plus upvotes. I think we made the front page of Reddit. It was it was pretty amazing to watch.

Cristina: Yeah, absolutely. Well—

Kamyl: So as a part of that, you know, I want to now ask you a few questions. You know, that was sort of the tip of the spear when we saw this incredible engagement online. We thought, okay, how could we start to extend the conversation and maybe do some paid channels? What was our approach there? How did we start to pivot a little bit?

Cristina: Yeah, well we we saw this amazing organic growth and wanted to help amplify it. Normally we spend months planning a campaign and making creative, but this was very much about time is of the essence and we had to mobilize quickly. In a way, we had planned for this kind of moment, seeing the big reaction in this yes/no AI campaign we did earlier this year, and so the foundation was in place. We launched so many channels in record time. A national billboard campaign, including over a dozen in Times Square, radio, TV, YouTube, many, many channels. And the messages included things like big tech forces AI on you. We make it optional. And AI surveillance should be banned. AI should be optional and private. We wanted to be very straightforward, point to the moment we were reacting to, and share a bit of a TLDR manifesto.

Kamyl: Yeah, it was so fun to see the team work so quickly in getting all this new content up and then seeing seeing the response. So can you talk a little bit about what the response and the results have been like?

Cristina: Yeah, I mean, on the qualitative side, like we saw in the organic response, it was just incredible. Folks echoing our rallying cry, engaging with the ads, talking about it in a way we haven’t seen before. Social ads were getting tons of upvotes and shares, right? An ad getting upvotes, it kind of hurts my brain a little bit. But as a marketer, it’s amazing. And lots of jokes from people about how they actually like seeing this ad. To build on what you said earlier, we saw tons of installs and visits with a combination of the organic and the paid work that we did and you know records for for us internally. Installs climbed to a peak of about 75% above pre-announcement levels. And you know, again, I’ve settled down, but a really healthy baseline, above baseline. And then visits to no AI also surged, peaking at something like 6x what we had seen. And then on the marketing side, you know, we measure a host of other metrics like click-through rates. Which in some cases were three times better, which to me really indicates that people are interested and want to.

Kamyl: Yeah, the the ads with upvotes I think broke a lot of people’s brains. And I think one of my favorite kind of anecdotal things is has been just hearing from people. My you know, uncle saw the news about you guys. This is awesome. And just sort of hearing people throughout the company sort of feeling the momentum in their friends and family. Why do you think there was such a strong reception to this?

Cristina: I think it’s the same reason that our yes/no campaign earlier this year resonated so much. It’s because just because companies can put AI everywhere doesn’t mean consumers asked for it. And while others are racing to make AI unavoidable, we’re focused, as you said, on making it private, useful, and optional. And our view is simple AI should be on your terms, not the platform’s terms. And so we were speaking to a large, overlooked community who previously didn’t have a platform. Or a seat at the table with a tech company, we acknowledge them. While not saying AI is bad, but rather that AI should be optional and we offer both experiences. A lot of people are interested in AI, but they still want simplicity and control and choice. And that’s the space I think that DuckDuckGo serves, because consumers shouldn’t have to choose between trying AI and protecting their personal information. And I think always private is a much better proposition than just trust us, which seems to be the approach that many other companies are taking.

Kamyl: Yeah. So do you think we’re gonna keep talking about this? I feel like that that’s not optional. But what is what is what is next for us on our sort of next things we’re gonna be talking about?

Cristina: Yes, AI optionality and our messaging around it is not going away. Something we’re gonna hold near and dear for a long, long time, forever. And in fact, we are about to launch a new campaign that we’ve been planning for six months now. Maybe it’s already launched by the time you at home are seeing this. And in it we underscore optionality. So that if you are interested in AI, you can use it privately without having to worry about where that very personal information is going. It comes to life in a very visceral, cinematic way, and I’m really excited for everyone to see it and to continue to live in this moment that DuckDuckGo is having and for more people to come to the duck side.

Kamyl: You know, I hope people keep their eyes peeled wink wink for what’s to come. And thank you, Cristina, for this episode of Duck Tales and thanks everyone out there for listening. Looking forward to talking to you soon. Bye bye.

Cristina: Thank you.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com

Beah: Hello, welcome to another episode of Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. In each episode, you’ll hear about our vision, our product updates, engineering, or approach to AI. In this episode, you are going to hear about our voice AI chat feature. And I have with me today Fran, who played a very large role in building and shipping that feature. So Fran, do you want to just briefly introduce yourself?

Fran: Yeah, so hi everyone. I’m Fran. I’m an engineer at DuckDuckGo and I’ve been working on Duck.ai for a long time now and yes, I was heavily involved in building voice chat and yeah, it’s a very exciting feature.

Beah: Awesome. Well, thank you for joining us. And I’m Beah. For anyone who hasn’t already met me on one of these podcasts, I’m on our product team. Before, so I have a bunch of questions for Fran, but before I jump in, I want to make a ask of our listeners, actually. So we’ve been doing these podcasts for a little over six months, I think, now, and have a pretty good audience following on Substack. We would love to hear from you, our listeners, get some feedback about like things that you you like about how we’re approaching this, things maybe you don’t like, and also like what topics you would like us to talk more about so we can plan future episodes with your interests in mind. So if you have any feedback or want to talk to us, send us a note at podcast@duckduckgo.com. and we’ll also put that address in the show notes. So with that said, let’s talk more about voice chat. so Fran, can you just tell me like to get started, what is this feature? How does it work? Maybe you can give us a quick demo.

Fran: Yes, definitely. so voice chat is a hands-free conversation mode inside Duck.ai. so yes, so far our users have been able to talk with any with many chat models via text. And we wanted to provide a privacy focused way for you to just have a conversation with your model. There’s some really cool real-time models out there right now, and we wanted to incorporate this feature on Duck.ai as well. so yeah. With voice mode, you can basically have a call with the model and talk directly to it. It will listen, it will reply out loud. you can also interrupt it mid conversation and like if you want to correct or actually add some extra information, the model will stop and listen to you and then reply back. It can pick up a little bit how are you expressing yourself to also help that play into the response that it will give back to you.

Beah: They have people.

Fran: And so I have here on my phone Duck.ai. And if you want to use it, like we have this very handy voice chat, you can go there and you can just start talking. So can you give me the weather for New York today?

Beah: Yeah, yeah.

Fran: Just new it’s city in New York City, the temperature is around twenty eight degrees Celsius with a heavy thunderstorm. Celsius and the low tonight could drop to twenty-four degrees. There’s a heat advisory in effect as well, so it might feel quite humid. Make sure to stay hydrated and be prepared for possible flooding. You can also check the extended forecast on sites like AccuWeather or the Weather Channel. Yeah, that’s very nice. So yeah, you can ask follow-up questions inside the same session. Right now I’ve muted myself, so you can also mute in case you want the model to not hear you at the moment. And and yeah, it has access to web search as well, so you can also get the news, get weather information or any up to date information.

Beah: Nice. I am not too far from New York City and I can vouch for it being very humid today. cool. Yeah, so t tell me, I know that the demo, you know, I hopefully people can see it well enough f on the screen, but if you couldn’t, there’s like a little ghosty character with a bow tie. tell me a little bit. Who’s that? What’s his deal, his or her deal?

Fran: yes. so that is our voice mascot. we wanted to provide like a visual aid and like represent that the AI is speaking to you so you can just see when the AI is listening and replying back to you. So we created this mascot to be there with you during the session. And we wanted this mascot to also have a little nod to Dax, our technical mascot.

Beah: Yeah.

Fran: so that’s why it has also the green bow tie and it’s white, so it it it’s it’s a nod to Dax. Yeah.

Beah: Yeah, so if you’re not familiar with Dax, that’s the name of the DuckDuckGo duck that you see, the Debonair, yeah. duck with the bow tie. awesome. and by the way, like on the point about the dem demoing this is a little bit challenging because of the audio and the back and forth, but just go like after this podcast or in the middle of this podcast, go try it. it’s you know, if you don’t already have the DuckDuckGo app, you can download the DuckDuckGo app on your phone. You can also try it on desktop.

Fran: Yeah.

Beah: And actually, Fran, I should ask, is it free or do users have to pay to use voice chat?

Fran: It is completely free. so our free tier users can access voice chat directly from either that button or the sidebar as as you could see. And yes, with with the pro and plus subscriptions you do get more time to speak with the model, but it’s available for everyone right now on your phone on or desktop.

Beah: Awesome. cool. So maybe can you tell me a little bit about why we decided to build it at all? Like what instigated this whole project?

Fran: yeah, so like I was saying before, like we there’s some cool voice models out there and we wanted to give like this hands-free way for people to talk with with our with their chat model. and we also wanted to have a private way for them to speak, so we did put in place some technical ways for us to improve like our privacy and anonymizing your voice when we when you’re connecting to the provider to speak with it. and yeah we want to give you this free hands-free way of talking with the model is also better for accessibility like we’ve gotten feedback like some people struggle using the text chat so with this you can just connect and start talking with the model uninterrupted.

Beah: Even the. Yeah. Okay.

Fran: So yeah, it’s been a very, a very cool feature. And we know voice is a very unique thing for the users, so we wanted to provide a way for our users to be able to use that in a private and secure environment.

Beah: Everything. Okay. Yeah, yeah. Can you is there anything else that you wanna add about like how we handled priv like how we made it in fact meaningfully private?

Fran: Yes, definitely. it was challenging because right now there is so many different ways to handle voice models that are out there. Like different providers have different ways that they provide. Like we did this connection directly with the model using real-time web technologies. So you’re actually like having a call with the model. but there are other ways where you could parse the conversation, maybe turn to text, but then a lot of the details and the nuances of your talk can be lost in between. So we found we looked into this balance of having a good user experience while also preserving your privacy. So the way also that we build, like there is this relay in between you and the provider, and we anonymize all the metadata and the information of your call. So the provider doesn’t really know who you are. Although your va your voice is still unique to you. You should be mindful of that. But apart from that, everything else we anonymize during the call. Yeah.

Beah: Hmm. That’s cool. just to like kind of loop back to what you were saying about the why we build it in the first place. I’m personally like I’m not a huge voice input person. I know a lot of people are, but I don’t usually talk to my phone unless I’m actually calling a human. and I found it pretty cool ‘cause like, I don’t know, it is really convenient sometimes just to not I don’t know, they’re like when you’re on the go or something, right? And don’t have your typing fingers available. I’ve I’ve actually found that pretty handy and just been like impressed by how slick it is with the like hand it handling interruption. And I know we had to dial that in a little bit, like at least on my iPhone, it was way too sensitive at first and it thought like all the background noises were interruptions and yeah. But I I think now it’s pretty slick.

Fran: Yeah. No, it’s super cool and we’ve gotten so many positive feedback like from I’ve got people who reach out to me and say like I was stuck in traffic so I wanted to check something real quick so they use voice mode so they could still pay attention around them and like while they were sitting there waiting for traffic to move, they could quickly use and make some lookups. I’ve heard from people that also use it to practice a different language since the model can understand so many different languages. And even me, I’ve tried like speaking in one language and then switching to my native, like speaking English and then switching to my native one, and it can pick up like flawlessly. it’s it’s very nice. Yeah, I I enjoy a lot the the hands-free and versatility of it, and I’ve yeah, I I’ve used for a few things where

Beah: Nee. Yeah.

Fran: Me speaking is easier than me typing down what I want to to ask the model about. Yeah.

Beah: Yeah, I’ve been using it like I guess as like a thought partner. Sometimes if I’m like walking my dog and I’m thinking through something complex, it’s the kind of thing that like if my husband were walking alongside me, I would start just like, you know, spewing thoughts at him as a sounding board. I start spewing it at you know, Duck.ai instead. And I don’t know, it’s been kinda cool.

Fran: Yeah.

Beah: t tell me, Fran, like do you have a either a favorite thing or a favorite use case about about voice chat?

Fran: Yeah, so I I’ve used it a little bit in the past to help me when I need to write a write down like some idea that I have in my mind or like formulate some something that I want to share with a colleague but I really cannot put into words but I can express myself. So I would open voice chat and then like speak for a whole minute saying, I want to say this, but not exactly that word, but so like you’re saying, like this thought process. And then at the end I say, now summarize what I’ve said in two sentences, and it has been like really helpful to help me like consolidate a lot of flying ideas. And with web search that we added after the release, I also find it really nice, like to get, give me a summary of the news in my city today, or do these quick lookups where I can have it on the side speaking to me while also doing multitasking and doing something else. So yeah, it’s it’s been pretty nice for those use cases.

Beah: I mean I could. Nice. Awesome. Well, anything else that you wanna add before we wrap?

Fran: just want to invite everybody to use voice chat. I I’m biased, but I really like this feature. I I find it really cool, really like our ghosty guy. And and yeah, please test it out and share any feedback that you have with us. We are always looking for ways to improve our features and voice chat and Duck.ai in general. So, yes, please hit the share feedback button and we’ll definitely appreciate.

Beah: Does the ghosty guy have a name?

Fran: No, we didn’t give a name. Maybe if if anyone suggests a one that we like, maybe.

Beah: Yeah, I think you should have naming privileges Fran, so think about it.

Fran: Yeah, I’ll think I’ll think about some ideas. Yeah.

Beah: We Yeah, okay. Sounds good. I was gonna suggest Fran, but maybe you don’t want Fran. Yeah. Cool. Okay. Awesome. Well thanks so much for for joining and and congrats on this neat feature that you launched.

Fran: Ghosty Bow tie. Thank you so much.

Beah: All right. Talk to you later, Fran.

Fran: Take it. See you soon.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com

Beah: Hi, welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering, or approach to AI. Today I’m here with Matej, and we’re going to talk about image generation in Duck.ai. I will let Matej introduce himself, but quickly I’ll just say that if you haven’t met me yet, I’m Beah, I’m on the product team here at DuckDuckGo. And yeah, Matej, take it away. Tell us a little bit about yourself.

Matej: All right, Matej. I’ve been at DuckDuckGo for a year and a half now, and I’m working on Duck.ai and I work primarily on image generation last year from September until Christmas.

Beah: Awesome. Well, okay, so just to kind of get started, tell me a little bit about what that even means. What is image generation in Duck.ai? And if you’re able, I I think a demo would be great.

Matej: Yep, definitely. Let me share my screen here. All right. So if you go to the Duck.ai, go to the Duck.ai and you will see basically this screen and you probably notice that there’s this option new image, which is when you click on it, then you type any prompt in here. Duck.ai will try to generate an image for you. So let’s show me an image of a cat on the windows. So hopefully at the end of this we’ll get an image. This is going to take a little while because image generation is quite a heavy process. Yeah, this is the simplest form of it.

Beah: Okay. I see it. It’s coming together.

Matej: We’re getting, yes, it’s coming together and here’s our little cat. can click on it, zoom on it, download the image and copy it if we need it. Or we can keep it writing on it and ask for adjustments in the image.

Beah: That’s pretty good. Good. I mean it has like the right number of limbs. There’s nothing too crazy going on here. I’m pretty impressed. do you wanna demo like any adjustments or anything else you wanna point out while your screen’s being shared?

Matej: Yes, that’s correct. Sure, we can try it. Add another cat to the image. Let’s see what it comes up with.

Beah: Okay. another cat. yeah. Cool. And what what model is operating here?

Matej: So, we’re currently using GPT Image 2 model. We’ve been through multiple iterations of this and it’s always been a trade-off between cost, the output quality and the performance, so how quick the model is. We went through GPT Image 1, 1.5, 1 Mini and 2. Currently the GPT Image 2 model is slower, however it... the output quality is much, much better than previously. You probably cannot see it in images like this. However, if you would prompt it to generate a text, then you would see a clear difference between different models, and GPT Image 2 follows instructions really well and generates the text really well.

Beah: Hmm. Got it. So like a bad job if you say if you tell it to put some copy in an image is like it doesn’t get the letter. It like uses different copy, it like mangles the letters. Is that right? Yeah. I’ve definitely been there. But hey, look, this cat also has the right number of limbs. I don’t see anything problematic or fishy about this. I guess it’s kind of a s a softball query, because there’s plenty of cat images on

Matej: Exactly, exactly. That’s the case. Yeah.

Beah: In in Windows online, huh? Yeah. A demo query. Nice. Cool. okay, awesome. Well tell tell me a little bit, like stepping back again, tell me a little bit about why we even built image generation.

Matej: That’s correct, yes. Yes. So I think from the outset, we knew that we will have to at some point. The reason for that is most of the competitors or similar products already provide this feature. We knew that we had the capability, but also we had some user feedback that this is like a highly requested feature. It doesn’t match. Like at this point in time, this is like a baseline feature that is expected by all the users for you to have in any sort of AI product, I would say.

Beah: Yeah, yeah, got it. and all right, so what were like did you run into any challenges when you were building this this feature?

Matej: So I think challenges were not technical. Surprisingly, most of the challenges would come out of, for example, content moderation. How do we prevent people from generating, for example, self-harm images? And then the second part that I’ve already mentioned is how do we balance the quality, performance, and cost at the same time? So those were the two challenges that I would say were that come to my mind as we’re building this.

Beah: Yeah. Gotcha. And like what was the what was the runtime on this work? Like how how long have you been working on this?

Matej: So I started working on initial prototype in September last year, and we shipped it to production just before Christmas. So that was it.

Beah: Got it. And was it mostly you or was there a team of people working on it?

Matej: In the beginning, was mostly me and then another engineer joined me for UI work. So I did mostly the back end and then a little bit of wiring on the front end. Specifically for like, you can also upload your own image and then ask the model to make adjustments to it. So this is the part that I was working on.

Beah: Yeah, yeah. Nice. Have we gotten any interesting or surprising feedback since it’s been live?

Matej: So I wouldn’t say any interesting, but what was surprising to me was that the reception was quite positive. And I was a bit worried when we built this feature because in general, image generation is perceived as a fairly harmful thing. depending on your intent, you can also do harmful things. However, I was surprised that most of the people were positive and they appreciated to have a privacy focused way of generating images.

Beah: Yeah. Mm-hmm. Mm, sounds like the hard work that you put into, you know, putting the right content generation guardrails on it and so forth must have must have paid off.

Matej: I hope so.

Beah: Nice. So like going forward are there improvements that we’re that you guys are making on this feature?

Matej: So we’ve already made a bunch of improvements. This current iteration that you saw in the demo is an evolution of original implementation. Previously, we had a separate mode just for image generation. It was not integrated into your normal conversations. And this is one feedback that we’ve received and we pivoted away from the separate mode into having it in line in your regular chats.

Beah: Mm.

Matej: Another piece of feedback we’ve heard, and I already mentioned this, we’ve gone through multiple models because people are asking for better output. Going forward, I don’t think we have anything in particular. if you have any feedback, I’m happy to read through it and put something on the roadmap. There are many ways we could take this. I can imagine we have like we had we can add a freeform canvas and you can ask a model to make your image pretty like you draw something with your free hand and you’ll get a nice image or advanced editing you mask out something and you ask a model to replace that bit of an image with something else. So there are many, many different ways we can take it.

Beah: Yeah, yeah. Nice. Yeah. going back to a second for the whole the like to the inline versus a separate separate mode thing. So that’s like just just in case it’s not obvious that like it used to be that you had to start your chat in image mode. And if you started with any other model or you know, you were mid-chat, you couldn’t like jump into image generation, right? So now if I’m like mid-chat on something, can I just ask it to generate an image and Like it’s properly it can jump between those modalities.

Matej: That’s correct, yes.

Beah: Nice. And then how does it so since we’re using one model for image generation, if I start chatting again, how does the how does the does it just go back to the previous model that I was chatting with?

Matej: Yes, that’s correct. So if you’re in a conversation, there’s in the background, there’s essentially two models. One is for your conversation, which will be GPT-5 mini or Opus. And then there’s another model that’s specifically for image generation that’s always being called whenever you ask for an image, which is GPT Image 2.

Beah: Got it. And I can just jump back and forth willy-nilly, yeah, to edit my image or have a chat. Okay. Gotcha. and then how do we know whether like how does the model know? Not not how do we know, but how or how does Duck.ai know whether to be you know, interpreting my prompt as a an image generation command or continue discussion?

Matej: Correct. Yes. That’s an excellent question. So we have image generation tool on our back end and basically it analyzes all the prompts that are coming in. And if any of those is categorized as one that is requesting an image, then it will trigger the tool and that tool will generate an image for you. So we have some.

Beah: How good are we at that? Like is the just are we v are we perfect at classifying or do you think there are times right now when like somebody wants to be editing the image and they get a text response or vice versa?

Matej: I don’t think we’re perfect, but I think we’re pretty good, I would say. I would say about 90%. We have a set of evaluations that we run on these tools and on various prompts, and we keep adding to the dataset that we have. So we keep improving this and it’s already pretty good.

Beah: Yeah. Nice. okay, one more question for you. What is your favorite thing about image generation in Duck.ai?

Matej: My favorite thing, I think, like I had a good time with my wife when we were just goofing around. were generating, I think we generated like a demon llama in a coat or something. I don’t know, like things that you would never think about, but it was, it was just funny to keep working on the same image and what it would come up with. Another use case that I really love is generating logos. If you have an event you want to put on your, like a logo on your invite, then it’s...

Beah: Mm.

Matej: It’s pretty easy. It generates you in generally good enough result for you to use.

Beah: Yeah, yeah. Yeah, I actually I had a good logo experience with Duck.ai not too long ago. I had I was talking to a a friend, a a child of a friend, who with my daughter had they made this like amazing like role playing card game and named it and they wanted to create a logo for it and I was like, yeah, let’s let’s just see how it does. And I just on my phone, we were like literally on a hike and I just picked up my phone and told Duck.ai what the g the can the game was called and like a few you know, suggestions for what this person had in mind for the logo and like what came out was was pretty great. I was I was impressed. It was like a it was neat that you could do like that, you know, literally like while hiking through a park somewhere in a matter of moments.

Matej: Mm-hmm.

Beah: So I think he was impressed too. Nice. All right. Well, anything else that you want to add, Matej, before we close?

Matej: Try using this feature. Give us your feedback so that we can improve it, we can keep working on it.

Beah: Yeah, for sure. We actually read that the feedback. All right. Well thanks for joining with today and thanks everyone for listening. See you around.

Matej: Yes. Thanks for having me, see ya.

Show notes: Shop the collaborations here

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com

Gabriel: Hello again. Welcome to Duck Tales, the show where we delve into all things DuckDuckGo. I’m Gabriel. I do a bunch of these, but not all. I’m the founder of DuckDuckGo. I’m here today with Dan. Dan, you want to introduce yourself?

Dan: Yes, absolutely. Hey everybody, I’m Dan. I help lead the brand strategy team and I’m excited to be chatting with Gabriel today.

Gabriel: Cool. So we’re gonna actually turn it around today and Dan’s going to interview me a bit about something special we’ve launched recently. That said, before we get started, we have a new email podcast at DuckDuckGo.com where we would love your feedback about Duck Tales. What episodes should we run more of? What do you like? What do you don’t like about the show? Should I get a real microphone? Blah blah blah. Please email us. We’ll we’ll read it and get back to you. But now we’re going to turn it around and Dan, take it away.

Dan: Amazing. Before jumping into the questions, I’ll first just introduce this special topic we’re gonna dive into. Today we’re here to talk about the how the DuckDuckGo brand is doing collaborations. And we’re doing branded collaborations with Atoms Shoes, with Cotton Bureau Shirts, with Yubico Hardware Passkeys, and more collaborations to come in the near future. So before diving into the specifics about why we chose each of those collaboration partners, we want to take a step back and chat about why, in general, we’re doing collaborations. We want to chat about brand and the DuckDuckGo brand in the larger landscape that we live in, and just have an interesting conversation about this exciting new work stream we’re doing. So with that in mind, I can jump into the first question. The first question is why go physical now? And to add a little bit more meat onto the bones of that question, allow me just to explain in a little bit more depth. It’s really interesting to see how some brands are bringing gaining cultural influence in unexpected ways. For example, a brand like Teenage Engineering makes audio hardware, but has become a lifestyle statement. A brand like Starface makes acne patches, but it has become a signal a symbol of empowerment. And so that brings us to DuckDuckGo, which is an internet privacy company now releasing branded collaborations with fashion and technology hardware companies. So the first question is, why does it make sense for DuckDuckGo to go from a purely digital brand to one with a physical presence in people’s lives?

Gabriel: Yeah, and and if everyone is aware, Dan is relatively new, so he doesn’t know all of DuckDuckGo history. And I this is not as thought out as you might imagine. But let me take you through some of that history. So when when we first started, it was just me and I we’ll get into this a little more deeper, but I really like offline stuff. I mean as most people do, but like particularly like good products and showcasing

Dan: Yeah.

Gabriel: stylish things and whatnot. So I very early on made stickers and t-shirts. And for several years I would send people stickers for free. You could there was a there was a like a place on the site you could put in your information. I’d only use it to study the sticker and delete it in a privacy respecting way. But I probably sent liter I probably mailed thousands of individual people stickers out. At some point and people love it. And then we also had you know I I almost exclusively wore t-shirts for a long time and I I really had a thought, we’ll get you into this a little bit as to like what makes a really good t shirt. So I had some really good t shirts made and some we had a little store going that was just a cost and we gave away a lot of free t-shirts too. But at some point the free community picked up on this kind of stuff, which is not DuckDuckGo users, just people who want free stuff, and they overwhelmed my inbox and our free program such that we had to like completely take it down. And then we ran the store for a while with just the t-shirts, but our vendors kind of changed, went out of business. Various things happened where the store went down, and then we were like it’s not we got other things to do, so we haven’t put it back up. And so that was probably six, seven years ago, maybe when it last existed. So we did have it for some period or some amount of things. They weren’t collaborations per se, but DuckDuckGo offline products. And then it went away. Since then people continually asked for it. Can I get some DuckDuckGo stickers or t-shirts? And we have them internally, obviously, for company swag, which you probably have gotten already. And so we do send them around occasionally, but they’re not publicly available. Then fast forward to this year, and I just was like, I found myself with some extra time just looking at lots of interesting products for my personal consumption. And I was like, I would love to have a DuckDuckGo collaboration on some of these. And so these are the first ones we did. And I just wanted to exist in the world. And so it was it wasn’t some grand strategy more than I would like it. And then given that I would like it, I thought maybe other people would like it. We did do a lot of data research around this. And so our goal was to put it out, and hopefully our DuckDuckGo fans are interested in it. But that’s that’s really the longest word of it. Not a big strategy, more just like I want it to exist, and I’m very glad that products do exist and we’ll talk about them, but that’s really what it is. If if people really do like it, we’ll do a lot more. But if they don’t, then we’ll probably won’t do a lot more.

Dan: Well, I I think that makes total sense. And I also think that the timing for these products makes a lot of sense as privacy becomes goes from what is often considered more of a niche concern to what has really now become a really important mainstream concern for millions and millions of people. And so I’m curious when people wear the DuckDuckGo brand on a t-shirt or shoes or any other product, what does it stand for and why are you excited for people to rep the duck the DuckDuckGo brand?

Gabriel: Yeah, so we have a we people don’t really know because we’re not the majority search engine or you know, obviously that’s Google, but like in the US we have about ten percent of people who identify as DuckDuckGo users in the surveys we do. So it’s it’s a really not insignificant amount of people. And you know, our search share is lower because they may not use us all the time or on all devices, but that’s about the percent of users we have in the US. Those people, you know, are are what we call or we call deeply care and act on privacy people. They not only care, a lot of people care and say they care, but they’re doing something to act on it. You know, usually to escape creepy ads or just to protect their personal information from getting out, not leaving as much of a digital footprint, not having as many, you know, recommendations and algorithms, all sorts of good reasons to choose privacy, but they’re making that choice explicitly. And that’s kind of a lifestyle choice. And I think that lends itself to expressing yourself that you know you’re in this choice, like you care about privacy and you want other people to know about it. That also helps DuckDuckGo for everyone listening because most of our growth, the biggest component of our growth has been word of mouth. And so if you are a DuckDuckGo user, you know, you’re already educated enough to like make that choice and know why. The majority of people aren’t. And so it really helps like individual people to go out and educate people why privacy and and security are important online. And having something like a t-shirt or a sticker is like the icebreaker in that conversation. People wear the shirt and there other people come up. We get this all the time. People come up to them and be Well, what DuckDuckGo? Or or because we have a lot of users now, I love DuckDuckGo. And then they can talk about it and just meet other people who are also users. But I think that’s really the core the core like reason.

Dan: Absolutely. And I you’ve touched on this just in what you just said. But I know that wearing my company swag, whether I’m going to the gym or walking around the park, inevitably a few people come up to me and say, I love your shirt, man. Where can I get one? I love DuckDuckGo. What’s going on with DuckDuckGo today? And I explain, I actually work here, but soon enough we’re gonna have collaborations that everyone can wear and be a part of. It’s rare to see a brand where there’s so much true love for the brand and trust for the brand. And I’m curious, where do you think that brand love comes from?

Gabriel: I don’t know exactly, but I think it comes from or starts from that we really do put users first. I mean, like that is the goal of the company. We’re obviously like protecting your privacy, but even like almost more fundamentally than that, like we’re and it got started as a you know user first brand. So like we don’t make as much money as we could. That’s unlike most any other company because we are explicitly choosing to put users first and their privacy first at the expense of profits. And we think users appreciate that. In addition, we just try to have fun. So like the goal with kind of some of these collaborations and our kind of brand refresh we’re doing this year is just to make everything a little more delightful, like we want people really like our products. And so the people working on our products are every day thinking about how to make them so other people like the people using them actually like them. They’re not just utilities, even though they are utilities, but we’re trying to add little fun things in them. You know, we’ve had a show about the Easter egg logos that we have. So like I I think that all adds up to trust and kind of brand love in the end.

Dan: Amazing. And one more question before we get into the specific collabs themselves about our brand in general and how it relates to the collaborations. One thing that’s really interesting is there’s a lot of conversation happening around privacy right now. And there’s all sorts of word of mouth that’s being driven about our brand right now as it relates to privacy. And I wonder if you could just talk to why this moment in time is so important to believe in privacy and why folks might want their collaborations as a result of

Gabriel: Yeah, I mean the the really the need for privacy online has not diminished at all. It’s only gone up and up and up over time. It’s been in and out of the news for different reasons, like it was high in the news when Snowden happened at Cambridge Analytica, and then kind of fell off of it when the pandemic happened and then this rise of AI most recently. And but it’s always been there in the background. There’s still been data breaches day after day, and privacy invasions and the need, and the algorithms are even more invasive. So the need has never gone away. But it hasn’t necessarily been the dominant news story for the last couple of years. That is starting to change to your point. And that’s because AI, which has been the dominant news story, is very bad for privacy. All of the search privacy and browser privacy issues we’ve talked about for like 15 years now are all there with AI and even worse. Like you are giving even more explicit personal information to AI chatbots about very specific things in your life. And so the idea that those can be used to target you or used against you or be subpoenaed by a court or all or or leaked, like all of this stuff is starting to happen and will just happen more and more. And so the need in the context of AI in particular is very high. I mean in that context is why we offer like Duck.ai as our private AI alternative. But I think in to your question, the conversation is starting to to tend that trend there. And like this AI backlash isn’t about a lot of different things, but one of them is about lack of control and privacy in all these products that are kind of being shoved down people’s throats.

Dan: Absolutely. I think that makes a lot of sense and segues us in nicely to talking about some of these products themselves. So let’s first start with the YubiKey. Gabriel, could you tell us a little bit about the YubiKey, why it’s a good fit for the DuckDuckGo brand, and maybe even show it to us if you have it on you?

Gabriel: Yes, so YubiKey is a is a more niche but important product for people in general. It is it looks like this. I have it here. It is a key, almost like a USB key that in the way that it looks, and in fact this is USB-C and they send it to you with these stickers if you buy the DuckDuckGo cover. So there’s like a our logo on one side and then the Duck shield that’s protecting you on the other side, and then they actually come in a pair of two. So the other one has this our fire icon. A security key is people are getting more and more familiar with the passkeys, which is like alternative to passwords on your device, like you can use your fingerprint. A security key is like that, but it’s physical. So when you log on to a website that’s really and you wanted to make it the absolute secure, you can use one of these physical YubiKey security keys and you put it in your computer. You can leave it there all time. And then when you log in, it asks you to tap it. Or it’s it just recognizes the key and you can log in. So the difference is is like, you know, your passwords can be stolen. If you have the security key, it’s on the security key. So you the security key itself would have to be stolen. So you keep this at your house or in the keychain or safe, and you can have comes in a pair, so you you have two, so there’s a backup. I use these for bank accounts and things that are real secure accounts. And it’s the recommended approach to be the most secure with which a lot of our users appreciate. So security is not exactly privacy, those are two different things, but it’s very complementary. And so we thought it would be a natural fit.

Dan: Very exciting and works really well with a lot of our privacy features that we offer here at DuckDuckGo in a in a in both in the free version and then also it builds on a lot of the offerings we have in our subscription as well. So just a wonderful physical addition to increase your security online. That is probably the most utilitarian of the collaborations we’ve done. I will now move to the second one, which is a Cotton Bureau shirt. Which I think all viewers looking at this can see Gabriel is wearing. But Gabriel, do you wanna tell us a little bit about Cotton Bureau and why we’re excited about this shirt?

Gabriel: Yeah, so I like I mentioned wear a lot of t-shirts and had always been on the search for like what is the best base t-shirt material that I can find that’s like soft but not too thick and like doesn’t wear out the design on it and just fits really nice. I ultimately found this like Next Level, which is like the manufacturer base shirt. And try to figure out who sells that with different designs. And that’s how I actually came on to Cotton Bureau. Cotton Bureau itself is a also a Pennsylvania. We’re a Pennsylvania company. That’s where I live. They’re also based in Pennsylvania, but in Pittsburgh, we’re outside Philadelphia. And they specialize in kind of using this base shirt, but then partnering with designers across the world to make bespoke on-demand, cool t-shirts. So you can get like basically one of the kind kind of they they’re small runs of really interesting shirts. Both for brands, but also just just random designers are uploading cool designs there. So you can find some just some really cool shirts. So I both that’s how I came to them and I bought tons of cotton shirts over time. But when we came to you know make their DuckDuckGo shirts, blank. They’re the best shirts. So like we give these out to people, we can get out of swag for years. And I don’t know, I can’t tell you how many times people are like, that DuckDuckGo shirt you gave me was like the best shirt that I own. So people wear it a ton. Now this particular design and this one from the YubiKey, we like the idea was that we would for this initial batch of stuff make a theme, and that theme is dark mode. Which is and a lot of people on DuckDuckGo use dark mode. You don’t have to use dark mode. But it’s available. I use dark mode associated with privacy. So this shirt is, you know, obviously a black with our tone-on-tone logo, which is kind of looks cool. And that’s the same logo on the the version on the YubiKey.

Dan: Amazing. And for folks who may not be as steeped in DuckDuckGo lore, can you tell us a little bit about the duck on the shirt that you’re wearing and what that represents?

Gabriel: Yeah, so this is our mascot. His name is Dax Brown. The Brown part is a homage to Back to the Future for anyone who is familiar with that, the character Doc Brown. But anyway, his name is Dax Brown. He’s normally outfitted with a green bow tie, and you know, with an orange background if you’re familiar with our color logo. But we also have lots of other logo versions. This is our kind of dark mode swag version that we have been using for company swag for many years, and so we thought we would offer it to the public in the in this collaboration.

Dan: Amazing. And then that brings us to our final collaboration in our initial launch, which is our Atoms shoes. Gabriel, can you tell us a little bit about those?

Gabriel: Yeah, so I also have them here. This is a shoe and it has a little Dax, you know, dark mode version on the side, on both sides. This is the left and the right. I love these shoes, so this this is originally why I wanted to do the whole collaborations to start with, was to make a version of these shoes. I’ve been wearing these shoes since they came out really in 2019. The story of Atoms is really interesting of itself. It’s two people from Pakistan who really wanted to make like the best everyday shoe, but they started there and emigrated to America. There’s a on their website there’s a long Humans of New York story about their history, if you’re familiar with that. So I encourage people to go on and check that out. But the shoes themselves are just they really nailed it with the best everyday wear shoe. They are just super comfortable. They have the elastic laces, so they just slip on. You never have to tie them, but yet they don’t come off. The soles they made are just really soft and good for walking, and they’re very breathable. This is all like breathable canvas. And I just like wearing them. I wear just I walk around a lot and they’re great. And then they’re also just quite stylish in the sense that they’re both casual but like not casual, not too casual to wear to almost any outfit because I don’t I’m not a big dress up person so I feel I can get away with wearing these pretty much everywhere. So yeah, these are basically the same shoes that I generally wear except with a little DuckDuckGo logo on it. So I that’s why I really wanted to make them.

Dan: Amazing. Well, for listeners, as you can see, these are all products that Gabriel has been very involved with and can vouch for personally and enjoys. Gabriel, you chat chatted about this a little bit, but for folks who don’t know exactly what dark mode is, which is the theme of all of the collaborations, can you just explain that to us?

Gabriel: Yeah, so it’s it’s been so interesting how dark mode has become mainstream over time. So I’m I’m super curious actually, what percent of people don’t know at this point, but like it’s a good point. So like if you go now into any operating systems like your iOS or Android or or Mac or Windows, you can set the system theme. They often ask whether you want the light it to be kind of a light or dark. And I think most people still choose light, but a large percentage of people now choose dark. So it’s I think the last time I measured is on the order of like 30% or something like that are choosing dark. When you choose dark, you know, the system is generally just instead of like a white background by default, it becomes a gray or black background. So we at DuckDuckGo have offered dark mode as a theme well before this got popular, because you know, on computers like Linux and stuff when you use terminals to code, the default was always black. So I think that’s how it originated as a mode and then it became called dark mode. But when you do dark mode, you usually want to theme the website or system. So it’s not just the background, it like other things look good when when you change the background. And that’s how it becomes kind of a mode. So this is kind of an homage to all that for people kind of who like dark mode or who just want a darker item, which usually goes with more things, it kind of works better in those scenarios.

Dan: Yeah, absolutely. It looks great. It’s also an extension of the digital experience. And we’re excited to be launching more in the weeks and months to come. That wraps all of the questions for this episode. Anything else you’d like to add, Gabriel?

Gabriel: No, it’s been great. Thank you for turning around and interviewing me, Dan. Just to remind people that we have this new email address, podcast at DuckDuckGo.com. If you didn’t like this for whatever reason, or you did, or you have some suggestions for new episodes, or we should go deeper into some topic or another, please email us. We will read it and we will incorporate your feedback. So thank you. And thanks, Dan.

Dan: Thanks all. Thanks, Gabriel.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Beah: Hello and welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering, or approach to AI. In today’s episode, we’re going to talk about user research. And so my guest here is Zac. Zac, do you want to introduce yourself?

Zac: Yeah, hi, Zac Pappas. I am on the Insights team at DuckDuckGo. I’ve been here for coming up on 14 years and pretend to participate in a lot of market research, user research, and other conversations that we have where we try to infer what users want and whether or not we’re giving it to them.

Beah: Sweet, thanks. And I’m Beah. If you haven’t met me, I’m on the product team at DuckDuckGo. All right, let’s launch into it. I have questions for you, Zac. Are you ready?

Zac: Ready as you’ll ever be.

Beah: All right, so let’s start. Why don’t you give me an example? Like when we’re talking about validating things with user research, just like let’s set up an example of what that means and what that would look like.

Zac: I think so. Yeah. So I guess starting with what is user research and why do we do it? I think the obvious answer that people tend to think about are, you know, sitting in an interview or doing some sort of focus group and learning from people. But really, for us, user research is kind of all of that plus, you know, internal dialogue and real testing like A/B testing. So to put it, you know, simply, I guess, user research for us is just trying to be wrong very productively and being efficient at being wrong productively over time. So for us, everything really starts out as a guess or an assumption about users or their behavior about, you know, what is upcoming in the market and research checks that, you know, our assumptions are correct or at least as close to correct as possible. And we try to do that as cheaply as possible, meaning quickly without spending more time on it than what we’re gaining in terms of the product that we’re creating. So an example could be our homepage at DuckDuckGo.com because our homepage has been around for as long as I think DuckDuckGo has been around, but has gone through many, many, many forms. We’ve done tons of A/B testing on it, showing one version of it to some users and a different version to other users. We’ve done a lot of user testing on it, like sitting down with actual consumers of the homepage or users of the homepage. Everything from discussing with parents how they perceive the homepage. So a lot of subtlety that goes into what we think maybe appears as a pretty simple layout or design.

Beah: I had a little connection issue there for a second, but I think it’s alright. Let’s forge ahead. Sorry, audience. So, the homepage, I think, as you pointed out, sort of like deceptively, seems deceptively simple, it’s not highly functional software, it’s a page, there’s a few boxes on it, a field, but like what makes it so tricky and worthy of user research.

Zac: Yeah, good question. Because it is kind of subtle, or I guess it doesn’t seem very obvious. So if you think about it, the homepage is really where we have a big mix of different user types coming in. Making any assumption about a part of our product usually means that you have to infer who is using it, meaning are they younger or older? Do they have a lot of context about what they’re doing or very little? So our homepage is actually a mix of different user types. There’s a lot of returning users who are very familiar with DuckDuckGo, they’ve been coming to it for a very long time. And so when they come to the homepage, it’s a pretty like rote act for them. The other group that really comes to our homepage a lot are new users or new visitors to the brand and product space in general. So they have maybe very little context about what DuckDuckGo is or what products we have. And so you can imagine, you have these two very diametrically opposite groups that come in, people who have a kind of a strong habit formation and muscle memory on how to use the page and then others who don’t have anything and they kind of, you know, maybe say meander or explore more than they are, you know, directly going to the thing that they know that they want. So what we learned over years of A/B testing, user testing, and kind of regularly reviewing what the experience is like for real people, we found something kind of interesting, if I can just generalize it, which is new users — like kind of people first visiting DuckDuckGo for the first time — really look at the center of the page. And if you see DuckDuckGo.com today, it has these two prominent boxes in the middle. It depends on the browser that you’re in or the form factor that you’re on. But primarily what we know is new users will tend, their eyes kind of follow the middle of the page. And so when they land, we really want to show them an overview or a good sense of what DuckDuckGo can do for them. Whereas returning users, people who’ve been coming back for a long time, tend to want to find the search box. And so we can, or at least we have some confidence that we can move the search box around, you know, the top of the page where it’s at now, to the bottom, to the middle. And if you’re a returning user, because you kind of expect or know that it’s there, you will more easily be able to hunt and find it. So the configuration that you see today, and maybe changes as things develop, really does to us strike a very good balance between new and returning user needs, both something that a brand new user can figure out and use efficiently and kind of get a lay of the land of what DuckDuckGo is and what we do. And then returning users can quickly get to the search box or other more mechanical parts of the product without too much friction.

Beah: So how did we actually learn that? Like what testing methodology did we use? Can you walk me through it?

Zac: Yeah, a lot of, I’m sure, pretty standard things for people who are used to product development, but A/B testing primarily. A couple years ago we were running, I think, like at least one A/B test of the homepage, like one alternative version of our homepage every week for pretty much the entirety of 2024, 2025. That year we had a ton and tons of homepage tests. And then independent of the actual A/B experiments that we’re doing, we will regularly run diary studies. So recruiting 10 to 20 DuckDuckGo users or other different user types, even non-users, to give them some background on the product. And then really give them a chance to use it for multiples of weeks and have them do journal entries and take notes of the things that they’re running into that they’ve experienced that are emotionally supercharged or maybe had caused some great duress in how they tried to use the product. So anything that’s really notable to that user can come up in a diary study. What we’ve also done in the past is to try to create alternative versions of our homepage from designs or a design system that we can give to users to play around with in real time. So we can actually simulate what an alternative version of the homepage might look like and how it should act and give them a clickable prototype that they can use, get some feedback before we ever invest too heavily in the platform. So it’s been a mix of really just doing a lot of testing pretty much every week since the conception of the company on some run. The homepage was a big focus for us for the last few years to try to nail something that could eloquently show what all of our kind of product offering is between search and now the expansion into Duck AI and of course the apps and email and other things.

Beah: Yeah, so like there’s like, sounds like we’ve been marrying, and I think we do this elsewhere in the company, a mix of like quantitative research where we get to hear how people think, you know, people who are opted into some kind of study with us, like we actually get to understand what they’re thinking, what they’re doing in some detail and then just like quantitatively looking at results in an A/B test.

Zac: Yeah, absolutely. And probably the maybe thing I didn’t say was we were very active on our social media. We point most of our users, at least if you click around our website, a lot of things point to our subreddit, which is reddit.com slash r slash DuckDuckGo. If you have product feedback, if you run into a product issue. So we do try to, and we have lots of people here that every day are checking comments or posts to that subreddit just to see if we released something and it’s causing some undue user harm. Tons of feedback boxes around our SERP, like the search results page within the app. And so we’re constantly trying to encourage that if a user has a negative experience, they’re able to get it pretty frictionlessly by a message to us. So we’re looking at social media, we’re really running really structured user tests, like intentional user tests, A/B testing and just kind of a what they call dogfooding or like internally using the product a lot and trying to be very critical about our understanding of what users really need.

Beah: Yeah, I’ve said it before and I’ll say it again. If you’re writing feedback to us, a human is probably not just reading it, but like taking it to heart and thinking about it. So, um, how about like, tell me what are the privacy considerations? Can we do these experiments in a way that respects user privacy anonymity? And does that throw any monkey wrenches into the testing mix?

Zac: Yes, truly. Yeah, good question. Yes, it has, it continues to, but we continue to have a lot of really intelligent people here that think and care a lot about it. So over the years, we’ve gotten a lot better at developing methodologies that allow us to consistently protect and respect user privacy while getting some of the analytics that we would want from robust A/B testing. So we do this through local storage or through consumer opt-in notices. So for example, if we have a survey that we show you on our SERP, we will have like a pretty robust disclaimer there that says this is anonymous or the responses maybe are going through a third party if it’s something like SurveyMonkey that we use for collection of survey responses. So very upfront to the user and very much building the engineering that is required to do anything that’s an A/B test or more like metric tracking in a privacy respecting way. And we’ve gotten a lot better at this over the years, but it definitely takes a lot of toil and discussion internally with data science and really good product folks like yourself to understand what things you just can’t get from numbers, which is a lot of like how the product feels to people and that’s where we come back to the qualitative research a lot. Getting, you know, FaceTime with actual people to hear what their experience is like and hearing in their words gives us the why to the what that we get from the metrics. So for as much as we can measure something, we don’t really know why it’s changing or what that we were making about a user behavior and how it’s actually playing out. So for those, we do tend to get or try to get face-to-face with users and recruit them either through our own product. If you see a survey and have the option to, you can connect with us at the end if you’d like to share an email address that we can reach out to. And again, those are all done within the in accordance with our privacy policy or the policies of the platforms that we’re providing them. So it’s been a long road, but we think we’re in a pretty good position now to be able to test most of the things that we would ever want to test with a pretty high level of accuracy.

Beah: Nice. You kind of touched on this, but like who’s the we? Like who’s actually doing running these tests, writing these questions.

Zac: Yeah, so when we started the user insights team, as maybe the long answer to this, and we started the user insights team, it was because everybody was doing testing at DuckDuckGo with product folks, engineers, and we were, as I’m sure you know, but for the listeners, like we’re a very validation heavy company and we care a lot about the assumptions that we’re making to make sure that we’re working smart and we’re a small company relative to the other larger competitors out there. And so we have to be a little bit more intentional and I should say, like, careful about how we approach things. And so when we started the insights team, it was really on the basis of everybody at this company doing some level of validation. And we wanted to up level or make sure that there was a baseline level of excellence that went along with that. So the short answer to your question is everybody here does some level of user testing. I think you yourself have probably done more than I have, at least in the recent past. Because it is a part of every conversation that we have. And so we have this formal team that we call user insights that tends to do the higher risk or the more robust or deeper research that takes somebody’s full attention for maybe multiples of weeks. But if you’re a member of the product team or engineering or you’re on the copywriting team, you yourself should have, I hope, the capability and tools to go out and do any validation that you would have to do today. So I can take a guess. There are probably five to ten people I can think are running an active test right now on something like usertesting.com or PickFu. or something that’s a pretty light way to free up a future discussion if you can provide some data ahead of that meeting or wherever that discussion is taking place.

Beah: Yeah. And one of things that I think is cool is the user research team, like part of their, the team members, part of their job is running the research, but part of their job is making other people, like enabling other people to run the research. So even like if I’m running a test, like I can get support or help from somebody on the team to like check it, make sure the work was good, to understand the tools that I need and so forth. Okay, I think that was a great, I bet that’s probably not true everywhere and a good, interesting answer, but before we close, anything we didn’t talk about that you want to talk about, especially anything that’s in the surprise column or might be unusual about how we work at DuckDuckGo.

Zac: I think there’s a lot of stuff that we do and usually at DuckDuckGo I think most of it’s probably for the better and certainly the thing that I can keep reiterating is just how much we all care about users. Obviously the, you know, dedication to privacy that we have comes from a place of wanting to be treated better as people and consumers ourselves but in the way that we approach research I think the heart of it is getting to how people really feel about the product and whether or not it’s a compelling experience. So for all of the metrics that go up or for positive feedback that we’re getting, I think we really strive to know why. And some of the interesting things, I was just trying to rack my brain earlier about some things that have come up that might be worth sharing. And certainly, like, you know, we have a bunch of layers around the company that do either social media listening or user testing. But where I think this really comes out is when we’re shipping a product, we have a final stage in our internal process where we can kind of all see the changes that we’re making. We call them ship reviews, but we tend to dogpile in and see UX changes or copy changes that we’re making and have an opportunity to really look for confounding evidence or a contradiction that might help improve that thing. And so this process that we have actually does involve more people at the end than it does at the beginning sometimes because we’re asking that if you can improve something or if you have positive or if you have any meaningful feedback about something to jump in. So something that I think people would be surprised to hear is that even though teams tend to work on individual projects here, anybody from the company can show up during the process of creation of product and leave their own feedback and have like a real impact on the outcome of what that project is doing or the direction that that product is going, which is I think really interesting.

Beah: Yeah, one thing that we’ve been doing. Yeah, no, go ahead. Go ahead. Zac, please.

Zac: Independent of that, we found a lot of the subsegment. I’m sorry. There’s a little glimmer. Sorry, you’re, yeah, you’re, you got it. You’re, you were frozen. You go. I’ll stop. I’ll start my next table.

Beah: Okay, the listeners are gonna love this part. I was gonna say in ship reviews at that sort of last mile before we ship, one thing that we’ve been doing that I think we’re building maybe more muscle around now that I think’s been really good is like if we have some internal difference of opinion on like the best way, you know, some detail that we’re like, oh, this didn’t make sense to me, but it made sense to me. User research team members or anybody really is like trying to jump in with some, as one of your team members coined it I think, punk rock research — like really quick, like you know three-hour turnaround, like let’s get some data points to help unstick the conversation and and like base this decision as much as possible in empiricism. And like, you know, doing perfect research is very, very hard, but doing like, you know, imperfect research with known caveats can be quite, quite fast and productive and I love that trend that’s happening now.

Zac: Yeah, yeah. And the tools are getting a lot better. Certainly, I think as we see AI develop and get more integrated and things become automated, things that we’re looking at now are things like synthetic user panels, which not to get too technical is just using AI to simulate as best as it can what a group of or a segment of people might say. And so with that, you can imagine doing a lot more testing or even having it automated so that you don’t have to do it and you just wake up to the results from them. But of course you have to make sure that the output of this automation or of the AI is actually reflective of the underlying user base that it’s supposed to be representing. And that’s like a constant challenge for us. It’s the same thing that we’re dealing with here. When we ask or get results from something, you know, we can’t take it as truth. We have to take it even still with a grain of salt despite the validation and continuously validate. After we launch something, you shouldn’t be surprised to see that even if it was really positive, we might come back and try to change it again a couple months later if we have a really good idea.

Beah: Yeah, or a couple of weeks later. When you’re talking about the synthetic user testing via AI, like user testing script, like what would you do on this screen, AI — like that’s a great question. Excellent point. I hadn’t thought of that. What I would do on the screen is…

Zac: Yeah. It’s saying you’re not trying to do something on the screen, you’re trying to do something on the internet. That would be the phrasing.

Beah: Alright, awesome. Well thank you, Zac. I think we covered a lot of ground. Appreciate it. Pleasure as always talking to you. See you around the DuckTales hood.

Zac: Thank you. See you at the office. Bye.

Beah: Bye.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Greg: Hello and welcome to DuckTales where we go behind the scenes at DuckDuckGo and discuss the stories, technology and people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering or approach to AI. I’m Greg. I work on the search engine here at DuckDuckGo. With me today is John. John, would you like to introduce yourself?

John: Hi, I’m John. also work here at DuckDuckGo on the search engine and I’ve been focused on working on local search for most of my time here.

Greg: Awesome. Yeah, that’s kind of what we wanted to talk about today is local search. And so I’m looking forward to chatting with you a little bit about how local search works, how we do it at DuckDuckGo and get into some of the challenges that we have as a private search engine and how we solve those. Local search generally, it’s kind of maps related searches. whether you’re searching for places, businesses, restaurants, those kinds of search queries we kind of call local searches. We have a couple different versions of a map that shows up on the search results page. When you do one of those searches, either you’re searching for something kind of in a city and you put the name of that city in your search query, or if you search for something near you, have ways to provide those results. still in a privacy preserving way. So maybe as a starting point, I thought it would be good to just, if you could walk us through a little bit of the life of a local search query, how it goes from kind of something the user has submitted to us to what we decide to show on the page.

John: Yeah. So we start with a, we get a user query. So somebody type something in and sends the query to us. And we try to figure out whether that would benefit from some sort of mapping information. So maybe the query contains a number and a few words and then drive or street or road. And you know, that looks like an address. So we’ll show a map of that address. Other common patterns are, you know, common. types of businesses like restaurants, hotels, cafes, and those sometimes come in with location indications. know, cafes in New York City or cafes in Paris. And sometimes they don’t, sometimes it’s just cafes or cafes near me. So we kind of have to handle both those types of situations. And the ones that are nearby are really kind of the more interesting ones. because we have to find a way to figure out where Nearby is for the user. And we have to do that in a very privacy-respecting way because that’s our whole thing. So the browser, the web browser that you’re using will send a location. But we don’t just straight up use that because we deal with lot of partnerships where we have to get some data from our partners at Apple and TripAdvisor. And we don’t want to just send them raw user locations because that is a privacy issue. So what we do is we’ll kind of add some noise to it. So if you say you’re at, you know, this exact location in a city, instead of sending that, we’ll move you a little bit. Depending how big the city is, we might move you just a few hundred meters. If you’re way out in the country though, where maybe you’re our only user for miles, you know, we might move you a few miles. And so that... can affect your results, makes them a little less accurate, but we think we do it with as little noise as possible to get you a lot of privacy benefit while still giving you results that are hopefully very accurate to the location you’re interested in.

Greg: Yeah, so there are a couple of things in there that I think are interesting to talk a little bit more about. when you kind of described how we add noise to a user’s location, there’s a lot of documentation, by the way, about this, if you want to read more on our help pages. But I’m wondering how we think about the kind of trade-off between making sure that the results are still good and the way that we’re obfuscating someone’s location is sort of sufficient from a privacy standpoint. So maybe you can talk about the first part first. How do we think about the quality of the results and whether or not we’re showing the right local results for someone when they do one of those nearby queries?

John: Right, yeah, we have a lot of criteria that we check against before we show results to users. We’ll get a whole bunch of options that might be relevant to the query. And if you’re searching for a cafe, maybe a restaurant is relevant, but probably a hotel is not so relevant. And so we may make sure we get the category right first. And then we also want to make sure the location is correct. And because we’re dealing with partners, they might give us some locations that aren’t as relevant just because we don’t give them as accurate of a location. And so we’ll kind of refine those on our side without sharing some of that information with a partner to say, you you gave us this location that’s 100 miles from the user and that’s maybe because we gave you the wrong location on purpose. And so we’re not going to show that to the user, but we will show these other locations that are actually nearby and relevant to the users.

Greg: Got it. Yeah, and then, I mean, even, you know, sometimes when there is a location in the query, it could be one of those that is still ambiguous to us, right? I think a lot about the difference between Paris, Texas and Paris, France. Users don’t always clarify what they mean. So we have to try to figure that out based on context too, right?

John: Exactly. even some locations that are very well defined. If somebody says New York City, that’s just one place that we’re going to look. But New York City is huge. just determining which part of New York City is maybe the most relevant to send to the user is another challenge that we have to figure out.

Greg: Yeah, and I mean, just we’re sort of riffing on these now, but, you know, if you search someone searches for New York pizza, right, that could be they’re looking for a pizza restaurant in New York City, or they’re looking for a restaurant near them that serves a New York style pizza. So, yeah, all kinds of things that we have to sort of contend with and try to figure out to give those right results. I’m wondering if maybe we could talk a little bit more than about. the different types of ways we actually display the results once they appear on the page because we have a couple different versions of our local modules that we show depending on the type of result and the number of results we get back. So I’m wondering if you’d say a couple things about that.

John: Yeah, we have many different ways to display based on what the user is searching for. As I mentioned earlier, we have a special address display if you’re just looking for a map given a location. That’s a pretty bare result. It’s mostly just a map and the location label. Some more popular landmarks, we have a special display for that where you’ll get the map and if it’s the Eiffel Tower or something like that. Then you’ll get a picture of the Eiffel Tower and a little description that we sourced from. Wikipedia and then there’s a business searches is the next big category and those are kind of divided into two as well if you’re searching for a very specific single business like, you know Joe’s cafe and Las Vegas I don’t think there is a Joe’s cafe in Las Vegas maybe there is Then then we’ll just show you that one Joe’s cafe ideally, you know, we don’t want to show you Bob’s cafe That’s also in Vegas or Joe’s cafe. That’s in Idaho or something

Greg: There might be.

John: And then there’s the sort of category or chain type businesses where if you search for Walmart or restaurants, we’ll show you a big listing of businesses and where they are distributed on the map to let you kind of click in and explore more if you want to do details about each one individually.

Greg: Got it. And then that’s gonna vary too based on the category of the result, right? I think we have some kind of special display for hotels as well.

John: Yeah, we’ve just kind of introduced this and I can share this real quickly because I think it’s a pretty cool feature. So just recently we’ve added some pricing indicators. So if you’re searching for a hotel, you probably want to go stay at one of these hotels. And one thing that is important about staying in a hotel is how much is it going to cost and is it available at the time I’m traveling. So we’ve got a new interface. We just released this last month where you can put in your check-in dates, check-out dates, how many people, and it’ll tell you a pricing estimate of how much it’s going to cost for those dates. And then once you click into it, you can get different offers from different providers. So you can get expedia and go through and actually have really easy access to booking one of those hotels that you’re interested in.

Greg: Awesome, yeah, I think this is a really good example of something where it’s a new feature that we’ve added that captures a particular use case that we know users have. mean, that we ourselves as users of a search engine have when you’re booking a hotel, trying to make it easier for users to complete the task that they’re doing, not necessarily just kind of see the different results. I’m wondering, maybe you could talk a little bit more about some of the other challenges around those additional use cases or business data that’s important for us to show and how we’re thinking about that and how we see that evolving over time.

John: Yeah, we build this feature on a few different really important partnerships with Apple, Yelp, and TripAdvisor being the main three. And we use those partnerships to give us all sorts of data about different businesses. Yelp gives us lot of information about restaurants and new service businesses, shopping, stuff like that. And then TripAdvisor gives us hotel information and information more about that. attractions, know, museums, theme parks, stuff like that. And we’re trying to push more into using this data to give the user a lot richer experience. Some ideas, you know, including menu links for restaurants, giving the ability to order food or serve tables, call us a service business, you know, if you want to plumber, you can get it like kind of a quote directly. And all those are kind of ideas that were pushing for and hope to release soon.

Greg: Yeah, definitely. There’s all kinds of different use cases that people have with businesses. know even just knowing the hours that the business is open, right, is so important. And these are things that users ask about all the time. I think if I had to pinpoint one big challenge with local in general, it’s just that the coverage for the business that the user cares about in that particular moment is so important. And so we’re constantly trying to figure out ways to expand that coverage and make sure we have the right information about as many businesses as possible.

John: Yeah, exactly. We’ve done a lot recently also in terms of taking user feedback because, you know, we found users get very upset if we send them to a business and we got the hours wrong and the business is actually closed or they try to call a business and it’s the wrong phone number. So we’ve taken a lot of steps to submit or solicit feedback from users on when we get it wrong and then forward that upstream to our partners so that they can correct it and we can correct it and everybody gets a better experience.

Greg: Of course. Yeah. Yeah. Yeah, definitely. I want to emphasize, if you as a user send us feedback that, know, hours are wrong or, you know, an address or a phone number, you know, we take all of that in and do our best to forward that to the source of that data and also make sure we’re not showing it to other users and kind of getting better and better at correcting that over time. Cool. Well, we’ve covered a lot of ground. Anything else you think is worth sharing and talking about with respect to local search?

John: nothing comes to mind straight away. I’m hoping we can keep working on the product, making it better. think we’ve got some good expansions in the future. Some more, pie in the sky ideas we have of, you know, moving to something like a flight search or something might, come into later on. and yeah, just hoping to keep making the product better and getting users what they want and giving them the right information as quickly as, as easily as possible.

Greg: Yeah, definitely. All right, well, it was really fun chatting with you about this today, John. Thanks for taking the time. yeah, maybe we’ll have a chance to talk more on a future episode about other search-related things. But yeah, thank you.

John: Thanks for your talking.

Greg: Bye.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Cristina: Hi, and welcome to DuckTales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering approach to AI, or how we operate as a company. I’m Cristina on the marketing team, and I’m here today with Baran on the data science. Would you like to say hi?

Baran: Hi everyone.

Cristina: So for our viewers, if you’ve already listened to episode 9 with me and Chuck, you know that, like our product philosophy, privacy is core to the ethos of our marketing. And most of the common marketing practices we just don’t do, identifying and targeting individual users, retargeting, using behavioral data, using third-party cookies and pixels, all that, all hard nos. We have very thoughtfully developed privacy-respecting measurement that is a bit unique, but includes techniques that I think and hope all marketers can adopt. Thankfully, we’ve come a long way from when I was doing the campaign analysis. And thanks to everyone’s data science background, we have much more sophisticated techniques and models. And I’d really like to dive in on a slice of that today. So first question, most companies measure their marketing through attribution, tracking who clicked on what and then installed. We sometimes do a version of when it’s possible to in aggregate and anonymously of course, but can you explain how does it work for us and is that even the right question to be asking?

Baran: Yeah, thank you, Cristina. On one hand, we have a self-built, simple and privacy-respecting attribution. And on the other hand, we have a few channels who do their own attribution. But with both of them, we have three issues. One, only a small percentage of our total impact through the campaigns is captured by attribution. Let’s take the most obvious one, TV campaigns. There might be a QR code on the screen, but most people would just go to the app or Play Store to download the app instead of scanning the QR code. And then it doesn’t work on all channels. Many channels only support the industry standards, more privacy in ways of attribution methods, not necessarily what we built. And third and most importantly, which is also valid for all other companies, is attribution is not the same as incrementality. It measures who clicked, but not which channel actually caused, let’s say, a metric like an install. And the real question is actually incrementality.

Cristina: Okay, so then if attribution doesn’t answer the real question, how do we introduce incrementality and how does that answer the question?

Baran: So ideally it would be A-B testing for incrementality for a channel within a channel’s advertising platform. But there is no way to do this. Channels don’t provide this. It could also be because this would make it very easy for marketing teams to understand the incremental impact. The closest thing we can do is randomized GeoLift tests.

Cristina: Okay, so incrementality is a concept I think a lot of marketers are likely familiar with, but to make sure everyone’s on the same page, can you walk us through what one of these tests actually looks like for us?

Baran: Yeah, we can take all the geos in a country, for example, if it’s the US, we have the DMAs, they are like New York, Los Angeles, we would randomize them into two groups of test and control. While doing this randomization, we will optimize for comparability between the groups in terms of the metric that we’re interested in measuring, such as installs. And then once we have run the test, let’s say the test would be launching a campaign in the test geos, we would use the pre-period of the campaign to measure the relationship between these tests and control groups aggregated metric. And then for the campaign periods, we would use that relationship. And then based on the control geos actual metrics in the campaign periods, we would estimate what we call the counterfactual, what we expect the metric in the test group to look like in the absence of a campaign. And the difference between the actual test results minus this counterfactual would give us the causal impact.

Cristina: Okay, I think that’s pretty clear. When the test is done though, what kind of answer do we get? Is it as simple as it worked or it didn’t?

Baran: Yeah, so it’s not necessarily binary significant or not. In this case, unlike product tests, significant would mean the channel had a significant impact on installs. But maybe it did, but maybe it was too expensive. What matters is, for example, metrics like a cost per install and especially an uncertainty around this metric. If the uncertainty is narrow, we would have high confidence in the result. If it’s wide, we would have lower confidence and take decisions based on this uncertainty. And we do these by using Bayesian methods, which is helping with better decisions than a simple yes or no.

Cristina: Got it. So if we think this is the gold standard, why isn’t everyone doing it?

Baran: I think the main reason is the operational complexity. A clean test requires ideally a full turn on or off in the test or control geos, and there’s multiple parties involved in this. Let’s say you have companies with a paid search team and you’d like to do a test on the paid search channel. It would mean they would have to turn off their channel and maybe they have goals that they have to hit.

Cristina: Yeah, this is an interesting peek into our culture. So can you help our viewers understand why we typically don’t have these problems?

Baran: Yeah, we have a small and unified marketing team. No one person or team is controlling a single channel. There are no, let’s say, siloed budgets per channel or channel group. There are no consequences if a channel stops. Let’s say if we don’t run a certain channel anymore based on test results, the people running these channels would switch to another channel, which makes more sense. And also lacking more sophisticated attribution make it even more important to run these randomized GeoLift tests. So in this case the necessity actually becomes a strength from the marketing data science point of view.

Cristina: I like the way you position that we turn a bug into a feature. We’ve had to get creative and develop marketing and data science methods that respect user privacy while still giving us the insights we need to make data driven decisions. I think it’s not just about what we do. It’s how we do it. That sets us apart. Our approach to marketing is as unique as our product philosophy. So what was it like the first time you ran one of these at DuckDuckGo?

Baran: First time it was also a new experience for me, because I was also used to from previous work the operational complexity part. When we first discussed about turning off a channel, I was very amazed when the team said, okay, let’s just do it, let’s do it like this week, next week. There was no conflict, even though it was a high scale channel, nobody worried about their goals. In the end, the results were not favorable. So we ended up significantly cutting spend and optimizing the campaign from those results, and since then we repeated this approach many times.

Cristina: I think that’s a great example. I like to think of us as a somewhat nimble team. And yeah, one of our core values is questioning assumptions. And this marketing experiment is a clear example of that. I really appreciate how we’re not territorial. We’re very much of all on the same team kind of mindset. So you get the results, then what happens?

Baran: So if it’s a new channel, let’s say we ran it for the first time, either if the results are favorable, we might scale it up, start running it further at scale, we might keep testing if the results require some optimizations, or if the results really out of hope, we will just stop further testing. And if it was an existing channel, let’s say we turned it off in the test geos, then depending on the results, if the results were not favorable, we might adjust spend, change our, let’s say, target cost per 1000 impressions or another metric that the channel allows, we might further optimize or if the results are more favorable than we thought, we might increase scale.

Cristina: Sounds like a lot of work. Do you have to keep running these tests every time?

Baran: Yeah, that will be very difficult, but quick answer no. Let’s say a channel, a digital channel has a meaningful attribution rate. Let’s say if a channel claims y attribution, it has, let’s say 100 installs. Then we run this test, we see, actually we measured 50. Then we would call this 50% the channel’s attribution rate. And as I mentioned, we were using Bayesian methods. We will also measure an uncertainty around this. Let’s say 50% is the point estimate. Our full uncertainty is from 40 to 60%. We would apply this ratio to ongoing attribution data that we get from the channel anyway. And we would monitor without retesting it. But of course, this attribution rate could be subject to change. If there’s a material change, let’s say a major shift in the creatives, a structural change in the placements of the channels in the age groups and so on, then we would retest.

Cristina: Interesting. So what about for bigger campaigns like TV, where it’s not possible or cost efficient to turn it on and off in specific regions?

Baran: Yeah, very interesting. Like if you’re running, if you would like to run long big TV campaign, we wouldn’t like to limit our scale by just using 50% of the geos or even let’s say using maybe a lower percentage as a control group could be very difficult due to how these channels operates. We use media mix models to measure the impact of such large channels. As other companies, we modulate spend from week to week. We have on and off periods, but the challenge there is TV is not the only channel that is running. We also have other smaller or digital channels that are too small to model within the media mix model directly. Here is again where the randomized GeoLift tests come into play. Same principle, we use a Bayesian model where we include digital channels using their attributed installs as the inputs. And we don’t need the model to guess everything. We already have our high-confidence measurements from the randomized GeoLift tests, and we can put those attribution rates along with the uncertainty as Bayesian priors into this model. So in one unified model, we capture everything.

Cristina: So then how do all the pieces fit together?

Baran: Yeah, this is, we are doing the same thing that the industry agrees. Attribution, randomized GeoLift tests and media mix models go in hand. One of them alone is not enough. Some channels have the attribution, but attribution is not telling the whole picture. GeoLift tests are great, but it’s impractical to random all the time. And media mix models are great, but many channels are too small to be measured there on their own. And we’re combining them all together.

Cristina: Well, thank you very much, Baran, for explaining part of how we measure campaigns. I hope it’s helpful and interesting to folks. And then more importantly, for your ongoing work, evolving our techniques to get higher confidence insights and decisions. We only spend marketing dollars on what actually grows the product, and we do it without compromising privacy. And frankly, we couldn’t do it without you. So thank you again, and we intend to keep you busy for a while. I so appreciate your time today and that of everyone who watched. So stay tuned for more episodes and bye for now.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Beah: Hello, welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, the people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering, or our approach to AI. In this episode, we’re going to talk about the fire button. And I will be your host today. My name is Bea and I’m on the product team. And we have as our guest Sveta. Sveta, do you want to just quickly introduce yourself?

Sveta: Yeah, I’m Sveta. I work on mobile apps at DuckDuckGo for now five years.

Beah: Sweet. Thanks. And you’ve been, you’re a designer by trade, right?

Sveta: Yeah. I’m on the design team.

Beah: And you’ve been doing a lot of work to improve the fire button on mobile. And so that’s what we’ll jump into today. Okay, so first of all, let’s start with the basics. Like, what is the fire button? What does it do?

Sveta: Right, so the fire button is a button that’s located quite centrally in mobile UI and it looks like a fire. And when you press it, it removes, it deletes all of your tabs and all of the browsing data and all of the history. So pretty much removes everything, all of the browsing activity and you can start from clean slate. And it also shows a really cool fire animation when you do that.

Beah: That seems like a good prompt to actually show it. So let me see if I can share my screen here. Can you see that, Sveta? Okay, then hopefully our audience members can as well. Alright, so this is the fire button here. I’m just on a website, on NASA’s website. I’m going to tap the fire button. And so I have, what are these two choices here?

Sveta: So the first, the primary button will delete all of the tabs, all of this data for all sites. And the second button is a choice to delete activity for only this one tab. So it will close this tab and will delete all of the site data associated with the sites visited in this one tab.

Beah: Alright, I’m going to choose that one here. Delete this tab. Awesome. All right. I just, I wanted to only delete the one because I also wanted to show like if you’re using, if you’re a Duck AI user, this is a new feature. But if you’re on a particular chat, for instance, a chat about what music your dog might enjoy, you can hit the fire button and now you have this opportunity or this option to just delete this chat. So remove it from your chat history.

Sveta: Nice.

Beah: Boom. All right. let me get it shared on. Okay, I don’t think we’re sharing anymore. Cool, so like what is the benefit of using the fire button?

Sveta: Yes, the primary benefit is privacy reasons, because as you browse the internet and you visit sites, you accumulate different site data such as cookies or different user preferences that then tracking companies can, if you accumulate a lot of it, they can use it to identify you and connect your browsing activity to you. Sell that information to other companies to show you targeted ads. And our browsers, we actually block most of the trackers and most of the cookies, but there are still some cases when we can’t do it. And then that’s a good practice to delete all of this data every now and then. And then the second reason is just still schooled and it makes you, lets you start your next session from clean slate.

Beah: Yeah, I like how those two, they’re like different reasons to use the fire button, but I kind of like how they converge too. Like part of what feels good about a clean slate and not having like stuff everywhere is, I think for me, like not having your data everywhere, you know? Like it’s just clean, it’s like a sense of control and I don’t know, care, I suppose, that you can apply more easily with fire button. Like you can tell me if this is how you would characterize it. Like you can delete data in most browsers, but it’s just a lot harder, right? Like you don’t have a button front and center with just the options that you need, like one or two taps away.

Sveta: Exactly. It’s usually tucked like hidden somewhere in the settings. I think it’s actually pretty cool that we managed to turn this really mundane and boring technical thing in a really satisfying experience.

Beah: Yeah, and one thing I didn’t show is that you can actually pick between, I think, three different animations. Like the fire is the default animation, but there’s like this kind of nice swirly animation and all like water animation. And then you can also just turn animations off if like what delights you is speed and efficiency and not an animation. That’s cool. Cool. Tell me a little bit about like how the fire button has changed over time.

Sveta: Yeah, so what the confirmation that we’ve seen just now, that’s updated design that we’ve launched recently. But for years, I think since we first introduced the fire button in our mobile apps, we just had this system, little system confirmation to confirm, delete all of your tabs and data that was very small in the bottom of the screen and it was really easy to press. And it was of course good for users who were regular fire button users because it was quick. But then there was also a big risk that you could just accidentally press it and delete all of your tabs and data without intent.

Beah: Yeah, so if your intent was to blow everything away all the time, fine, great, it works great for you. But if you wanted to be a little bit more surgical about it and keep some tabs around, then it neither worked for you. It didn’t really work for you, and if you accidentally used it, you might really regret it.

Sveta: Yeah, we did get some that obviously there’s if you’ve accidentally deleted all of your tabs without wanting you would write us an angry feedback, which we sometimes received you open it and you see all caps that chances are from someone who just deleted their tabs.

Beah: Yep. So take note, dear reader, if you are writing angry feedback, a kind human like Sveta is reading it and taking it to heart. And so in this case, what did we do about it?

Sveta: So we’ve changed a lot how the confirmation dialogue looks like. It’s bigger, that you’ve seen it. It takes a lot of space on the screen, so it’s harder to miss. We’ve on purpose moved the buttons a little bit high up. So when you press the fire button and your thumb is right above the fire button, the confirmation buttons, they are not on the same spot. And there is an animation, pictogram that also attracts attention. it’s just everything is made for you to pause, read the label, and make an informed decision. And then another big change is that we’ve added a second option because before you could have only deleted everything, all of your tabs and all of your data. And now we have a second option that you can delete just one tab and the data for that tab.

Beah: Yeah, yeah, I love, by the way, like, I’ve developed, you know, I’ve been using our mobile apps for a long time, but I’ve developed this kind of workflow recently now that we have the single tab, single chat burn, so handy, we have this kind of, we’re in the process of updating behavior when you open the app after an idle period, we take you to a new tab automatically, so you can just start typing your query or your chat or whatever. I like realize there are tabs that I want to keep around, the things I want to keep around, but there are a lot of things that I just want to do a quick search, a quick whatever, and it’s just, it’s throwaway. And like now I open the app, I’m in exactly where I need to be, I don’t have to think do I want a new tab or do I want to, you know, override this tab? And then like when I’m done, you know, five seconds later, just doop, tap, burn that tab, and it’s just, I don’t know, there’s, it’s a very satisfying and efficient workflow for me for the I don’t know, 70% of things I do that are just quick interactions with the app and don’t require any kind of historical retention. So I love it, Sveta. Thank you.

Sveta: I think there’s something interesting about this approach to delete activity from one tab, because in other browsers, the way they solve this problem, the task of deleting some of the data, not all of the data, they use time. So you can delete an activity for, let’s say, last 20 minutes or last hour, but then it deletes it across other tabs, all tabs. While in our browser, like you said, you just did one thing in one tab, and then we delete that one tab without affecting any other tabs.

Beah: Yeah, yeah. I think it’s a great shift. Do you want to talk at all about the design process that you used with the team to get to these improvements?

Sveta: Yeah, so our first hypothesis or first direction that we explored for this new mobile confirmation was the same approach that we have on desktop browsers where we have a list of options for users. Every time you press the fire button, you can choose, do you want to delete tabs or keep tabs, delete the side data or keep it. And that is a good direction because it gives most flexibility, but it also, we realized that in mobile context, just looks, it looks heavy. It took over the whole screen, the confirmation just, we were asking users to make these choices every time. We, well, in our old design where you had to only press one button. There was something nice about that privacy package. We gave users the most private option with one button pressed. So we decided to instead just add another option so you could either delete everything. With one button press or delete one tab.

Beah: Yeah, yeah, I think it’s like the fire button is such a great example of a feature that has like, sort of delight elements to it that make it really satisfying to use and sort of goes above and beyond like what, you know, the basic functionality that users might describe themselves as expecting from a browser, like, but it does provide this like really useful functionality. And it, I think it’s like the process to keep both of those things true, maximize functionality that people want and keep it delightful in its simplicity and not add cognitive load to the user experience is pretty tricky and seems like something that we navigated well in this case and got to a pretty good outcome, I think. Do you have, have we gotten feedback or do you have any indicators, how do you know if it’s a good outcome?

Sveta: Yeah, we do surveys for our new users and the last survey that we did, one of the questions, asked the users a specific question about the fire button and we’ve seen that the delight. I don’t remember exactly the question, but we ask if this experience is delightful. And that has increased compared to the old design. I think we will, I also see less angry feedbacks in all caps from users who deleted their tabs without wanting to. Something that was also a challenge in this project is that this balancing, making this confirmation easy enough for old users to not lose this effect of confirming quickly and deleting data with one tap, but also adding some friction to the new user so they will not accidentally delete it. Of course, we also received some feedback from the old users who thought the new confirmation was a little too complicated. But that’s the reason now think people are liking the new design.

Beah: Yeah, I feel like that’s an ongoing challenge with changing our mobile apps, right? It’s like we have this pretty big band of DuckDuckGo mobile app faithful who have a particular workflow that works for them. you know, whenever you change it, there’s some risk that it irritates people by kind of just like messing with their workflow, even if it’s a net positive improvement. Yeah, I think the fire button’s probably right at the center of that challenge because people who use it really love it and it’s like probably a big part of their experience of what makes the app special.

Sveta: Yeah, and we have a lot, have millions of users who use our regular fire button users, but we want more people to feel that this is something for them. I am actually, I wasn’t a regular fire button user because it was too distractive like my tabs, but now I also use it more often myself.

Beah: Yeah, yeah, same. Totally. rarely used it. Rarely did I want to destroy everything, and so same. I mean, I wish I were that person that was just like clean slate every day or every use, but of course I have several tabs sitting around that are supposed to be reminders for something, so yeah, I love it. Yeah, I was gonna ask you what your favorite thing personally about the fire button is, but you just touched on something you like about it now, which is per tab distraction. Anything else you would add?

Sveta: Well, when a few times when I do actually use the delete all app option and delete all of my tabs, I have to say it really feels nice. It feels like Marie Kondo, she came and cleaned all of my tabs and it’s a good feeling.

Beah: Yeah, so sometimes you look at your tabs and you ask yourself, do they bring me joy? And if the answer is no, burn them all down. Yeah, nice. So anything that you’re looking forward to, like any plans to continue improving the fire button in the future?

Sveta: Yeah, I think there’s lots of things we could explore and improve. We’ve maybe trying to see if we can find a new flow or simplify the confirmation for the regular fire button users. We’re also trying out something interesting on Android now where we’ve just going to launch to release this update. switching the buttons around and we’ll have burning, deleting the one top as a primary button. See if this would make this fire button feel more, less scary, less destructive and hopefully will make more people feel like that’s something for them. That will be interesting to see how that works out.

Beah: Nice. Awesome. Anything that we didn’t get to that you want to add, Sveta?

Sveta: I think, so I don’t think a lot of people know, but on our iOS app, you can actually customize that fire button. If you tried it and you’re sure that you want to use it regularly or not as much, you can long press and swap it for something else. You can also go to settings, appearance and do it from there. And if you do hide the fire button from the main UI, you’ll still have it in the main view. think that’s good to share. also, yeah, just to everyone, try the fire button. Give us feedback. We read it. It will help us. Yeah.

Beah: Yeah, nice. No need to use all caps, but we do read and appreciate the feedback. Yeah, awesome. Yeah, and if you were like a, if you discounted the fire button in the past because it was too destructive, let this be your sign to give it another try. Now that we’ve made these improvements to it, I think you might really like it. Cool. Well, thanks so much for joining, Sveta. I really appreciate it. And thanks to everyone for watching.

Sveta: can try with one tap, yeah, small steps. Thank you, was fun.

Beah: and see you later.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Beah: Hi, welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology and people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering or approach to AI. In this episode, we’re going to talk about our Sync feature. So today, Emanuele has joined me. I’m Beah. You haven’t met me before. I’m on the product team here at DuckDuckGo and Emanuele is, well, I’ll let you introduce yourself.

Emanuele: Yeah, thanks. Hi, Beah. Welcome everyone. I am Emanuele. I have been at DuckDuckGo for about six plus years and I’ve been involved in Autofill and Sync almost from the start. Yeah.

Beah: Cool. All right, well let’s dig in. I have a few questions for you. So first of all, what are we even talking about? What is Sync?

Emanuele: Sync is a feature that lets you synchronize your data from one device to another. For example, you may have a collection of bookmarks on your desktop and you want to see the same bookmarks on your mobile device. You set up Sync and all your data is showing across both devices seamlessly.

Beah: It sounds very handy. So you can sync bookmarks. What else can you sync across devices?

Emanuele: Yeah, you can sync also autofill data, credentials, and credit cards. You can also sync, well, we said bookmarks. Recently, we added Chat Sync, so you can sync your Duck AI chats across devices. This is currently only available for our own first-party applications, but we are expanding to other browsers as well.

Beah: Nice. So like, just to kind of paint a picture here, I have a MacBook Pro, I have a Windows laptop, I have an iPhone, and have an Android phone as well. And I’m like, you know, trying to log in to Reddit in all these places. If I save my Reddit credentials, my login, my username and my password on any of those devices, then it can automatically sync to all of those. So then the next day when I’m like out and about on my phone, if I logged in on my laptop, like the password just autofills and I can log right in. Is that right? And then now with that...

Emanuele: Exactly. Yeah.

Beah: Duck AI chats, as you’ve mentioned as well, if I wanted to like, so what’s the yeah, tell me about chats. What’s the use case there? Why do I care that it’s across my devices?

Emanuele: Well, the use case, I use it a lot personally when I chats on the desktop and then I want to continue in the evening on the couch, I can resume the same chat and continue the conversation with Duck AI from there. I also use it for looking at older chats and basically maybe I chatted with Duck AI for something a while ago, and all of a sudden I remember, oh, well, there was this information that I already discussed with Duck AI, and I can go when I’m on the go outside with a friend and show my finding even when I’m on the go. Well, with Duck AI specifically, the Sync feature also increases the limits of how many chats you can store in Duck AI. The limits used to be 30, we bumped it at 100, but with Sync you have a lot more storage, so you can store a lot more chats with Sync activated.

Beah: Nice, so if I’m like chatting on my phone, chatting on my computer, I’m like, you kind of need it because you potentially are like doubling or whatever, multiplying the volume of chats that you’re having and you don’t want to be like penalized, like lose those because they’re synced across devices. You just want them available.

Emanuele: Right. Yep.

Beah: Yeah, that makes sense. Tell me, like, let’s talk about how we do Sync from a privacy perspective. Like, are there any considerations there? How do we do it in a way that is not identifying and protects all of our users’ privacy?

Emanuele: Yeah. First of all, all of the data in Sync is encrypted end to end, which means nobody but you can access your data, not even us. We don’t have the decryption key, so the information stays encrypted on our servers and nobody can access them. We went a step further in that we don’t really ask you for emails to sign up on anything. We don’t ask you for your passwords or anything. We just set up an account in a way that’s secure with basically a random string that is used to encrypt, I’m kind of simplifying things, but like a random string that is used to encrypt your data and then the string stays locally on your device. This ensures that, you know, whatever happens in the server, the data cannot be decrypted by a malicious actor. And we, as a company, cannot know who you are. We don’t have any information on you and we don’t have any way to tie your data with a specific person.

Beah: How does that compare, like, how unique is that? Like, is that what everybody does with sync and password management or is that pretty unique and how so?

Emanuele: The end-to-end encryption is a pretty common pattern in password managers specifically. Not as much maybe in browsers, I don’t think. But the fact that the account is... that you don’t need an email to log into your account, that’s pretty unique. I don’t know that there are many other companies that do anything like this.

Beah: Yeah, I can’t think of any mainstream, well-adopted examples anyway where you don’t need an account. Cool. How hard has that been in practice? What technical challenges did it create? Or was it just hunky dory?

Emanuele: Yeah, no, it’s not easy, especially when you create an account that doesn’t have an email associated with it, it means that certain things that users are accustomed to, that they expect, they’re not there. So you have to kind of create user experiences that allow users to use their accounts seamlessly without thinking about an account, but with all the features that they expect. Specifically, the recovery part is the complex part because we don’t have data. We don’t have any way to retrieve your data. Therefore, you have to save a recovery code to decrypt the data on your side. The complexities that we face are both in terms of UX, as explained, but also there are complexities on the engineering side, as we need to ensure every time that all the flows are accounted for in this sort of accountless account. And yeah.

Beah: Yeah.

Emanuele: When we start to connect certain things together, like the chats and Sync and, you know, so we will look into subscriptions as well. We’re basically connecting different systems that all are sort of blind to the user concept, but they still have to sort of contain this data. So that’s a high level.

Beah: Yeah, as you were talking, I was thinking, this is like a great example of one of those things that’s like deeply, deeply complex. And like for it to work, you just have to like, you have to boil down that complexity and like, shield the user from that complexity and present a very simple, it works, very little room for human error, no requirement of deep understanding of how it works. And I think that is a really interesting challenge.

Emanuele: Yeah, I think our UX for setting up the account has this advantage that you don’t have to type anything and you just set it up and when you have to transfer it to another account, you just scan a QR code and the data is automatically transferred via that QR code scan. I think that’s kind of handy in the setup step. But again, it poses a few challenges then in the recovery phase.

Beah: Yeah, like you don’t have to remember a password or remember what email you used. You just have a device, another device, and then let them talk to each other. But getting there was really hard. I mean, and maybe there’s still places that we want to improve the UX, but I just, I know we’ve been like finessing it for a long time. Now I’m somebody who like has to sync things all the time because I’m constantly like uninstalling and reinstalling and trying new devices as part of my job, like testing and you know, understanding our products and looking for gaps and like, so I go through the Sync setup, you know, way more often than any normal user should have to. And it has gotten a lot better. Like I, there are moments when I’m like, that was easy, which is what we’re going for.

Emanuele: Magic.

Beah: Yeah, magic. Magic.

Emanuele: Yeah. We are actually working right now on an improvement to the Sync screens. We are going to launch an experiment on iOS first. And we hope that the Sync flow will be easier, even easier for users to understand specifically the very first step when you decide whether you want a new account or you want to connect to an existing device, that should be easier. But they’re just improvements around the existing flows rather than a new different flow.

Beah: Yeah, I think one of the things you’ve done really well as a leader of this work, Emanuele, is continuing to watch the data and poke holes in the flow and basically make sure that we’re simultaneously building new features and optimizing what we have and just making it better and better and better as we learn more. I’m curious if there are any kind of in that vein or you can leave that direction if you want. You can say whatever you want. If there are things that have surprised you in the process of building Sync and getting feedback and watching the data points and so forth.

Emanuele: Yeah, I think one sort of surprising thing was that the people were not really finding the feature organically. As soon as we started building entry points where they were useful to the user flow, meaning when you are saving a password, when you are watching a specific screen, like you are looking at your bookmarks, for example, and we show at that point a prompt that says, do you want to move this bookmark to your other device? And we do that in several touch points. And again, these have been studied to be the least intrusive possible, but also to pop up at the right moment when the feature is more salient to the user case. When we started doing these promotions, we call them promotions, we can call them entry points, basically making the feature discoverable. User adoption shot up quite dramatically and basically we were very happy to see the trajectory of adoption once we start to put in Sync front and center for the users. Basically it allows them to use DuckDuckGo, like enter the ecosystem and sort of buy into the whole system. And your data is there, your browsing is easier, your bookmarks are there, you can save something on desktop and again read that on your commute on mobile. Yeah, these entry points have been really effective.

Beah: Yeah, if you build it, they won’t come unless they see that it exists and you make it easy for them. Yeah, nice. All right. Tell me what, yeah, you kind of already touched on this, but anything that’s coming up, coming soon to Sync that you want to talk about?

Emanuele: Exactly. Yep. Yeah, a couple of things we are working on. The first is allowing third party browsers to sync their chats as well. This is the first time that we are actually building Sync on a third party browser that is not integrated with our apps basically. So that’s coming fairly soon. The new screens, we just touched on the new onboarding screen for iOS. And we’re also starting to sync subscription status. So if you have a subscription, a DuckDuckGo subscription on your mobile device, currently to move it to your desktop device, it’s quite a convoluted process. Whereas with Sync, again, we move to the QR code scanning. So if you have Sync already active, the data will start syncing automatically. And if not, you will be able to set up Sync again with a QR code right from the subscription screens and move your subscription information across very easily, which means your advanced model for Duck AI, they will transfer automatically and you can resume your advanced model chats across devices.

Beah: Nice. I am personally looking forward to that because I also go through the, you’re a subscriber, let’s retrieve your subscription flow somewhat regularly. Cool.

Emanuele: Yeah, it’s a bit convoluted. And we will also look into tab sync, which is a pet peeve of mine. I’m looking forward to that, which means that your open tabs on one device will be accessible from another device, you know, seamlessly through Sync.

Beah: Cool. Awesome. Well, I think we should wrap it up. Is there anything that you want to say that you haven’t already before we close? This is your moment, man. Go nuts. Yeah, that’s a good one. Set up Sync. It is awesome. I can vouch for that. Cool. Well, thanks so much for joining. Really appreciate it. Yeah, see you around DuckDuckGo.

Emanuele: You know, set up Sync, it’s awesome. Thank you, Beah. Bye-bye.

Beah: Thanks everyone.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Greg: Hello and welcome to DuckTales where we go behind the scenes at DuckDuckGo and discuss the stories, technology and people that help build privacy tools for everyone. In each episode you hear about our vision, product updates, engineering, our approach to AI from employees at DuckDuckGo. I’m Greg, I work on the search engine here at DuckDuckGo and with me today is Karl. Karl, would you like to introduce yourself?

Karl: Yeah, sure. Yeah, so I’m Karl. I’m a design engineer here at DuckDuckGo and I work across the search engine on instant answers and our Search Assist AI module.

Greg: Awesome. And that’s actually what we want to talk about today. DuckDuckGo Instant Answers, how we show instant answers for different search queries. Obviously, in a search engine, we show links to websites. But there’s a lot of other content that we show on the page as well, which I think broadly we sort of refer to as instant answers. You know things like if you search for the weather, you know, we might show a weather module at the top of the page. I’ll just share a couple examples of these. You know, so you search for weather and we try to show, you know, relevant information about the weather at the top of the page in addition to then all of the, you know, the links to different websites.

Karl: Nice.

Greg: We have things like unit conversions. If you search for a temperature and you want to convert it from Celsius to Fahrenheit, different calculations, all of these things are kind of within the umbrella of instant answers. We have had instant answers for a long time. I think really early in the history of the company and the product, recognized that when you’re pulling information from a lot of different sources, you have the ability to present that information in different ways to help the user find what they’re looking for, whether that is looking for a website that they want to navigate to, looking for an answer to a question, or looking for something a little bit more complicated. So yeah, we have lots of different instant answers, and thought we’d talk a little bit more about that today. So I guess as a starting point, I’m curious to talk about, and you mentioned being a design engineer, what are some of the challenges involved in designing and building instant answers?

Karl: So I think one of the things that I’ve always found working on these is it’s very easy to just basically take everything we get. So we will work with like a provider. Let’s say if we’re doing where to watch something, let me share my screen actually as well and we can have a look at this. We are working with a provider. So in this case, we’re working with JustWatch and using their API data. When you come to design something like this, it’s very easy to look at that and say like, okay, cool, let’s go to JustWatch’s website and have a look at what’s available. We can see their website, how they present the information they have and they give us. And you could kind of jump to say like, well, we’ll just take what they’ve done. They’ve got that data, we’re using the same data, let’s just copy the way that they’ve presented that information. That would be like the easy solution. But then you’ve got all of these sort of legacy things to deal with, like we talked about a bit earlier, we’ve had instant answers for a long time. We’ve got ways that we present our information on the search engine and we need these modules, whether they’re pre-existing or they’re new, like we need them to all feel like they belong together in some way. So it needs to sit in the search engine nicely, you know, and feel kind of like it belongs. So we can’t just do like a one-to-one copy, which, you know, as designers anyway, we don’t want to just copy what someone else has done. But when the data is so vast and so complex, it can be kind of hard if you’re not familiar with the domain so much as an individual to just figure out like, what do I even care about here? So like, here’s an example with a movie, we were trying to figure out where you can watch this, but like, where are you in the world? You know, because that’s like the first thing that’s probably one of the biggest challenges is we can’t show you the same thing if you’re in the US or the UK or Australia. You’re going to see something different.

Greg: I think you touched on a lot of important pieces to this that make this challenging. One is just understanding the user’s context. Obviously, we have limited information about our users based on our privacy policy and our privacy standards. We can approximate things like location accurately enough to be able to do things like the weather one that I showed, or this where to watch example. But that does limit us in what we are trying to do where we’re not trying to overly personalize the things that we’re showing down to a very specific user profile. And so, we have to kind of generalize and try to figure out just from the search query and that limited bit of information, what is it that we’re going to show to try to answer the user’s query. The other thing you touched on that I think is important to highlight is just the fact that this search query here is very specific and I think has a pretty clear intent where to watch a specific movie. Sometimes the intent is a little bit harder to unpack. And so you might have more different things that you could show. If you just searched for the movie title, for example, maybe someone’s trying to figure out where to watch it. Maybe they’re interested in the release date or different news about it. If it’s an upcoming movie, maybe they want to know what the cast is like. So trying to figure that out and then present the information in a way that gives users the ability to see what they’re looking for and also then refine their search down to the more specific thing. I think that adds some challenge as well.

Karl: No, I agree. I think probably one of the biggest things that I found difficult around this is exactly that is the like figuring out the query, figuring out the query space, and then defining the data set, the golden data set, as we call it, that helps us to understand when do we trigger? How do we decide what we trigger? And entertainment is especially challenging because we have three different modules that are part of the class of entertainment. We have the titles module, which is the overarching piece that tells you, you know, this movie is from this date. This is a brief synopsis and you can get to the two other modules. So this where to watch module. And then also you can get to the cast module from that parent module. But like you say, you have to trigger these differently and figure it out. And so we can do these really clear ones where someone puts the keyword in that we’re looking for, but sometimes we’ve got to try and use all these other signals from what we get from the organics or like other pieces of information that we’re retrieving to say like, actually we think this is best and you will know better than I, but like it’s incredibly hard and sometimes, you know, it doesn’t matter how hard you try, the intent might feel really obvious to the person typing it, but as far as we can derive from that information, we can’t always figure out the right thing to show you. And so we do our best, but you know, there’ll be misses sometimes.

Greg: Yeah, I want to ask you about something that’s on your screen right now because you can see, you searched the query Project Hail Mary cast and we’ve got the cast module at the top left there. We also have that Search Assist box to the right side that attempts to kind of answer the question. I guess, we’ve talked a little bit on previous episodes about Search Assist and what it’s doing. You know, how would you describe how we think about showing those two things next to each other, the different kind of roles they play, and maybe how that’s evolving over time?

Karl: I think, yeah, that’s been a big challenge for us because AI summarization is a really useful tool for some people. It’s not for everybody. And obviously, we’re super careful about giving people optionality and letting you say, I don’t want to see this. But for people who want it, people who are interested in seeing that, we don’t want to just always put that front and center as the key answer to every query. Because in a case like this, for example, Search Assist is giving you the same information, but it’s giving you it as text. And maybe we’re giving you a little bit extra too. But if I’m specifically trying to ask about the cast, there is a high chance it’s because I’m thinking of an individual who I’ve seen in the film, or I’ve heard is gonna be in the film, or what have you. And so I’m trying to see their face to be like, yes, okay, so it is Ryan Gosling who’s in Project Hail Mary. Okay, cool. And so in this case, like that is probably what we feel the most critical piece of information. So we want to give you that module upfront. But at the same time, if you’re someone who’s interested in getting a kind of AI summary as part of your query, we don’t want to then say, just, we won’t show you that tool then. And so we have this like convenient little right rail slot where we can say, actually, we’ll just move this over here. So you get kind of the what we think is the best of both worlds where we can deprioritize Search Assist in this case where the intent is really clear. And then in cases where we’re not so sure, we will do our best to try and figure out like maybe Search Assist is the best answer of this or maybe it’s a different module.

Greg: Yeah, I’ll also note, we have different sort of settings that you can use for Search Assist everywhere from, I never want to see Search Assist at all up through, show it to me as often as you can. You obviously have it set to show often here. I do too. But yeah, I mean, for this kind of query, certainly the use case you spoke to of like, I’m sitting with my family watching a movie, go, wait, who’s that? This is the type of thing where, okay, you can get that information really quickly. But I guess to your point, we recognize with a search engine, you’re walking a line between giving just the immediate information that a user is asking for and then giving them the ability to dive deeper, which a lot of people are using a search engine for. It’s sort of a, people come and type all manner of topics and things into that search box. And we kind of have to be ready for any of it and different user expectations. I would say we do, I’m curious to the extent to which you agree with this, like we do generally try to keep things relatively uncluttered, like we’re not trying to sort of throw everything at the page all at once. You know, I think from my perspective here, this is sort of one of the more kind of rich search results pages you would see on DuckDuckGo. You’ve got the news module down below. You’ve got some kind of imagery within the organic links. And that’s about it. You know, I guess from a design standpoint, how do you kind of see the role of just like the information layout and kind of how much we’re showing on the page at a time.

Karl: Yeah, that’s a good question. I’m a huge clean freak when it comes to design as well. It’s kind of hard trying to walk the line because when we talk to people who use the search engine, we’ll try to understand, how do you feel about the page they’re presenting? Especially if we try and compare to anyone else, what do you think about it? And we always get kind of mixed results, I guess, because some people love a nice clean layout. Some people want to see loads of stuff because they’re just so used to being bombarded with richness of like images here, there and everywhere. And one of the biggest challenges is that we obviously want to meet people where they are. We want to help give them what they’re looking for, but we don’t want to go to the kind of degree that we just say like, we’ll absolve all responsibility and just give you everything that’s available. Like here’s every image and every kind of video and stuff’s auto playing and like just taking up all of your space. Like for me, like you said, this current SERP that we’re looking at is this layout of the page. Like this is very rich for us. And it’s like at that kind of point of balance where I think we are okay with it. And we’re like, that’s good. But you know, if we imagined that Search Assist also had an image at this point as well and other things in there, it starts to make it really difficult for somebody to figure out like, where does my eye need to go, what do I need to look at in this particular point? So we’re trying to kind of keep it really clean and say, you know, cast is this most important piece right now. That’s you’ve got big images, big module, and then we kind of de-prioritize as you go further down. But it is probably one of the hardest challenges and it’s why whenever we’re developing a module like this, it’s not the kind of thing that we can develop in isolation. You know, you can’t sit in a Figma file just with this design of a flat version of a cast module and be like, you know, that looks great. I love it. Because the minute you put it in the SERP, you realize, you know, okay, I’ve got to deal with all of these other visual things and maybe we need to turn some stuff off or change something. And that’s I think, you know, being a design engineer here, it made it possible when we developing these entertainment modules for me to do that. So I was given the kind of the rope to get into the code base and start to put these things in a SERP and play with them. And I guess you just, you get a different feel for what works and what doesn’t work when you actually see it in real place in the code base.

Greg: Yeah, definitely. I’ll also add, similar to the Search Assist setting, we have a setting that lets you basically turn the instant answers off if that’s what you want to, recognizing that some people like more and some people want less. Although I think in a lot of cases, we see users keeping them on and finding a lot of value in them. I’m curious, as we maybe get towards wrapping up our conversation, putting your user hat on for a minute. What are the instant answers that you use the most?

Karl: Well, I will say that I use my entertainment ones all the time, of course, but that’s also just as a kind of, I guess, a curious person who works on them. I have to keep an eye on them. Search Assist is obviously one of my, again, most frequently used, but I think conversions is a surprising one that I use a lot, and I think that’s especially the case because being from the UK but working with a lot of my American colleagues, when people are just chatting about Fahrenheit, I have no idea. So I have to end up getting in there and converting that. But also great for time zones and things like that when you’re trying to figure out when you can do a meeting, you know, like this meeting, for example, I can jump into that. And then weather, again, like one of the simple things that just, it saves a ton of time when you’re already in a search engine doing other things, just pulling up the weather. IA is a super handy one. What about you? I feel like you’re gonna be a sports fan.

Greg: Yeah, well, we haven’t talked much about sports, but I do use the sports modules a lot. The sports ones are interesting because, you know, with something like conversions, you know, that’s pretty consistent as far as the queries we see day in, day out, people like looking for conversions. Sports is going to depend on, you know, the events that are happening, right? Our American football module is probably not triggering very much right now. But when the season starts in the fall, it’ll start showing again. And certainly, the day of the Super Bowl, it’ll be shown quite a lot. I’m a big Philadelphia sports fan. I have some queries bookmarked that I just look at pretty frequently. I find, personally, the instant answers for sports are actually kind of my favorite way to get some of this information. You know, if I’m looking for like the NHL standings and, you know, or kind of, you know, when do the Flyers play their next game or when did they play their last game? Like this search query is really quick for me to do. It loads quickly. I kind of know what all the information is here and where it’s going to be. So I find this pretty handy. And we’ve actually just this week, I think we’ve shipped some new sports. We’ve shipped Formula One, we’ve shipped NASCAR, and we’re planning to add a bunch more sports as well for people who want to get their sports information from DuckDuckGo. So yeah, I guess anything else we missed that we want to talk about before we wrap up?

Karl: I think we’ve covered pretty much everything, yeah, I can think of.

Greg: Yeah, I’m sure we could do future episodes deep dive into more. We didn’t talk about local at all, which is its own answer that has a lot of complexity to it. But yeah, I guess we can wrap up there. Thanks, Karl. This was fun.

Karl: Yeah, awesome, thanks.

Greg: All right, bye.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Gabriel: Hello everybody. Welcome to Duck Tales again. I’m Gabriel, the founder of DuckDuckGo. And with me today is Julia. Julia, do you want to introduce yourself?

Julia: Yeah, I’m Julia. I am part of the PeopleOps team. Been at DuckDuckGo for three years and a half. So yeah, almost four. Very excited to be here.

Gabriel: That’s cool. Yeah. It feels like a long time. I’m really bad at time. I know I’ve been working with you for a long time. You’re wearing a DuckDuckGo sweater from our... one more swag like that one. Yeah. Nice. Okay. So today we’re here to talk about hack days, which is something, it’s not exactly PeopleOps, but it’s something that you also just work on here. You’ve been responsible for it for a while. So yeah, tell us, tell us what hack days are.

Julia: Yes.

Gabriel: I know they’ve preceded you and I can talk about that too, but talk about your journey with hack days.

Julia: Yeah. So it’s, I love actually owning hack days because it’s not as... it’s related to culture, but not so specific to HR and PeopleOps. But hack days actually, it’s also known or mostly known as hackathons. A lot of companies in the tech industry do them. It’s a combination of the word hack, meaning creative exploratory programming, and marathon, which is something that you do fairly quick and in a short amount of time. So it’s kind of about working very intensively for a short amount of time and see what you can accomplish. It became popular in the late nineties, beginning of two thousands. And we just happened to call it a little differently. So we call it hack days. At DuckDuckGo, the way that we do that is about three or four weeks throughout the year. We do our hack days. It’s from Wednesday to Friday. So from Wednesday to Friday, we allow folks to just put all their regular work duties on the side, on pause, and using good judgment, of course, and work on anything they want that relates to DuckDuckGo. So full creativity, autonomy, and collaborating with other folks and in other domains. So that’s what it is.

Gabriel: Yeah, so a couple things with it. One is we, you do your largest part of this, so tell me what you think about this, but we’ve really tried to encourage people to do it too. So you don’t have to do it. I mean, that’s one thing, you can continue working your normal working day, but we’ve really tried to encourage as many people as we can to participate and also as we’ve grown to collaborate with others. With the idea being here that when you step away for those kind of three days, you know, without the constraints of like regular project scoping and oversight, and you just kind of left to like build from scratch, you can come up with and try new ideas. And especially if you collaborate and cross-pollinate, and as a result, I mean, we’ll talk about this, but a lot of good things have come about it. So we’ve been really trying to encourage people, but I think some people are kind of reluctant to do that. Worried they’re going to fall behind on their work or whatever. So yeah, so maybe you can speak to how we try to encourage people.

Julia: Yeah, it’s an interesting part of... So our culture has a lot of autonomy and one of our values is to build trust. So we treat people with trust, we trust folks when they are hired, we trust our hiring process. So we know they’re going to use good judgment, they’re going to put their work on the side if they can, they’re going to keep doing the things that they have to do and balance things out. But it is interesting because as folks come from other organizations, they do have... like some folks, some people have a fear of like, no, I’m going to leave my work behind. I don’t know how this works. Like, is it going to be okay if I actually put it on hold? Thankfully, we’ve been doing it for a while and a lot of people participate. So I feel like after you’ve gone through the first one and you see how many people participate, you’re like, this is actually a big thing here. And it’s cool to do it. So we try to motivate in a way that we build our culture, just like letting people know, like we trust you. We trust you and you can make that judgment call. We’re all grownups here. We all know it’s a business. We all know what we have to do. So it’s, I think that’s the way that we motivate. We also try to ask people to plan ahead. So we remind folks when we post the whole calendar of the hack days in the beginning of the year. So folks already know when that’s going to happen and they can kind of make sure that things that are more pressing in their regular work is taken care of in advance. And we announce a couple of weeks before it happens, so folks also have that just as a reminder. So those are some of the things that we do to motivate them. And they can also, if they’re not able to do the hack days that week, they can actually make it up to hack days in another week. So if that week actually just becomes something like, you know what, now I have to actually focus on my work, I won’t be able to pause things, they can do that in another week that seems more suiting for them.

Gabriel: Yeah, I was going to mention that if you didn’t, because it’s a subtle thing that we eventually did. I think another subtle one, but it’s just, it’s not even something we’ve done, but to your point, since so many people participate, is a lot of the leadership also participates. And I think, yeah, and I participate sometimes. And so when people see that, I think it may give more permission for people to do so, you know? Yeah, exactly.

Julia: You participate! Yeah, that’s an example thing.

Gabriel: And then on the other side of it, I know we’re gonna wrap this up, but maybe not as a good time, is like, you know, I agree, we have a good cadence now of like encouragement and lead up and stuff, but then the end of it is also kind of fun, as like kind of the end of the hack days. So what happens at the end of hack days?

Julia: Well, so we do hack days for several reasons, but one of the reasons is engagement. And one of the really cool things about hack days is you have from Wednesday to Friday, for those who don’t know, every Friday at DuckDuckGo, we have an all-company meeting. So we all get together. We talk about company updates. It’s kind of like what a town hall, a traditional town hall call would look like, but with our cool twist. So for hack days weeks, we actually try to, we cancel all the other meetings and that’s the only one that stays. And during that meeting, we have a showcase specifically for hack days. So that week is just about the showcase. We allow folks to talk about what they worked on for two minutes. We actually have to time it, which is kind of, it’s one of the things that...

Gabriel: We have a lot of people now. Yeah. Yeah.

Julia: I do not like. I’ve been really hard not to interrupt people and to just let them do it, but we are growing as an organization, so it becomes quite challenging as we grow the amount of hack days demos that we have. But during that call, you just get to listen to everything that folks created in three days. And it’s really fascinating because you get to see, well, for me, for example, I’m not in the tech team, so it’s really interesting to see how tech folks are thinking about the product, what are they doing to make things better? And from any other domain, you just get to see how people work and how fast they can accomplish things and how, you get to know more of the products because obviously they’re demoing things. So it’s kind of a win-win even if you don’t participate because you just get to learn a bunch and you just leave that call so motivated and so inspired because you get to see we are really a big bunch of like really smart people who are overachievers. So it’s really crazy what we can achieve like in three days. And it’s just like beautiful to see and you just leave that call feeling proud and feeling inspired and then you get your own ideas from it and you get to reach out to the folks who worked on things that you found interesting and just like talk to them about your feedback on that. So that is really special and I think that is the most special and impactful part of hack days, aside from, of course, the fact that it ends up impacting our products because several things are shipped and also internal processes are improved throughout those projects. So, yeah, it’s really exciting and really beautiful to see.

Gabriel: Yeah, agreed. I mean, let’s talk about that last part for a bit. So you can work on anything. We don’t even say you need to work on something that needs to ship. You know, like people can work on kind of pie in the sky ideas or little improvements they want to make. But some people do work on either internal improvements for like work process or development process or actual changes in the product that do end up shipping or spark ideas that end up shipping. And we do encourage that. I mean, so we had one, I’m just thinking of the Duck Tales episodes we did. We did one episode with, I think with Rachel, with the AI, no AI image filter that came out of hack days. Are there any others that jump out to you of like projects we did the last year or so that we ended up shipping?

Julia: Oh yes. I love that one. It’s so interesting that you say that because that is one of my favorites, basically, as someone who creates some imaging myself, as someone as an artist myself, I actually really appreciate that functionality of being able to filter out AI images and just be able to see what was actually created by humans. So I love that one. There are many, many things that I think were impactful. In the internal processes piece, it’s hard to say it because I would have to explain the actual process.

Gabriel: There’s been a lot of good internal improvements though. Yeah, like people make people’s lives easier. Yeah.

Julia: Yes, for sure. And there are small things like, for example, the automation that we did for when you’re out of the office, you actually get to update all your tools at once instead of having to go one by one. That’s really impactful and saves a bunch of time. We also had one for that you can just read the title of an Asana task in Mattermost, which saves a bunch of time so you don’t have to click to see what it is. So these small tweaks that are really important. From the product perspective, I think there was one about hiding distracting items from any website that I think is really, is very connected to what we do in our ethos of like user-first approach and giving people optionality. Also like vertical tabs. I feel like it’s one that folks ask for a lot. So that’s cool.

Gabriel: Yeah, the last hack days we had tons of vertical tab projects. We haven’t shipped those yet, but I think they will be coming and people will be very excited about that when we do.

Julia: Yes, we’re excited about those. I feel like it’s a big one and people love it. And I think, did the Easter eggs come from hack days?

Gabriel: Oh yeah, yes I did. Yeah, yeah, that’s true. I forgot about that. That was the project that I worked on.

Julia: That’s a favorite. Everybody loves it and gives a bunch of ideas. I think that’s really fun and very delightful. So yeah, some of it, but you probably know way more because you’ve been doing this for...

Gabriel: No, those are just a good example. So yeah, I mean, the point I just had was like, yeah, I think without really even trying, people really do improve things and we should kind of ship them and they make them into the product. I guess we could end with something about hack days are part of what we’ve been calling special days, or I don’t know if you’ve changed the name of that, but I think that’s what we still call it. And kind of like, it started with just hack days a long time ago before you joined, but then we had tried some other things too. Like we basically found that like, you know, about a quarter, four times a year is the right cadence for hack days, but there’s still some needs for other days to kind of take some time apart from regular work. And at some point in the past we had called them low-hanging fruit days or quick wins days. And it was kind of like these things that kind of fall through the cracks that maybe take just a few days or even you can knock out a bunch in a few days, but that we never end up doing because they’re small little tasks and they just don’t get prioritized. So that’s evolved, it’s evolved since you’ve been here too, but now maybe you could talk about what we do in the other months just briefly. Like I think we’re just calling this quick win days now, right?

Julia: Yeah, so for the sake of everybody who is listening, at DuckDuckGo we have something that we call special days. So special days is the umbrella terminology for hack days and quick wins. I feel a little bit bad for quick wins because hack days has way more hype to it. It’s a known terminology in other places.

Gabriel: Yeah. Just like this episode, we’re tacking it on to the end, but I think it’s still important just to mention because it’s an interesting concept that we’ve come up with.

Julia: For sure, for sure. So quick win days basically are things that you can do quickly. So it’s the same concept in a sense that it’s from Wednesday to Friday and you get to like pause certain work using good judgment and work on quick wins and things that you can accomplish really quickly or within that timeframe. I also do want to mention though that some things initiate as a hack days or a quick win. And then we realize the potential of that idea. And even if the person cannot end or finalize the whole thing in three days, we might just keep going because after the showcase, we realized the impact of it. So we have a very structured way that we work at DuckDuckGo. So having that space for creativity is really important. So going back to quick wins, those days are pretty much about getting low-hanging fruit taken care of. And a lot of people, a lot of our functional teams actually plan around it a lot. So what they do is that when they are working on specific projects and they see quick wins within those projects, they kind of just separate that for when we have the quick wins. And then they get a bunch of stuff done. So that’s really exciting because it becomes very productive. And it becomes a conversation within the teams. Like what are the small improvements that we can do in specific things or what are the little steps that we can go beyond in certain projects? So it actually becomes really impactful to have those days. So right now, as we evolved hack days and special days in general, we have tried to combine them. We have tried them separately. We have tried having themes connected to them. So at some point, the themes were connected to our objectives, which are our project roadmaps that are contributing to our top priorities and organization. So we have tried several things. Right now, what we do is that we have them separately, and it’s every two months. So basically, you have three hack days, three hack weeks a year and three quick win weeks a year. We try to skip weeks that are super busy like when we have our performance review process or things like that, people are involved in giving feedback. But yeah, that’s basically what those are. Also have recently implemented the... the difference with quick wins is that we don’t have the showcase, but we do tell people like, you worked on something really cool, do a demo. There’s always room for demos in our company meetings. So that’s really fun. And we are trying to increase that. And we also try to encourage people to work on not necessarily create something or work on something, but also learn something. So if there is a different domain that you want to learn about, or if there is... right now, we are highly encouraging people to test AI capabilities and what it can do, how far can it go? What are things that we can accomplish with it? What are the different tools that are out there that we can use? So quick wins also serve that purpose a lot. Like you don’t have to create something, just like learn, explore, do things that are going to help you do better work. So yeah, for hack days, we also implemented Hack Days Awards, which is one of the reasons why there’s a bit more of a hype with hack days than quick wins. So at this point we have an award, we have four awards.

Gabriel: You...

Julia: They are connected to what we’re trying to do with our product. So delight, dependability, and discovery. And then we have a fourth one for improving internal processes. And you, Gabriel, select the winners, which is really exciting for folks to just have that. And yeah, we give them a prize. We have swag that’s with special logos for our hack days. So that’s really exciting. We love swag at DuckDuckGo. So that is something that motivates people.

Gabriel: Heh heh heh.

Julia: Yeah, so I hope that answered your question about quick wins.

Gabriel: Okay, last thing, last thing. Yeah, it totally did. And part of the reason I asked was, I like about quick wins days is too, what you said is it’s not just engineering. Like it’s very clear that anyone has quick tasks across all the functional teams. Because with hack days, and this is maybe a problem with the name, like I think people assume it’s just engineering, but we have really tried to encourage non-engineering. And to your last point there, I’ll make this point that if you can... you can add anything for us to close out if you want. You know, we have been encouraging people to learn new things or use AI and I feel like AI has now really opened up in the last few days a clear path for even more non-engineers to get involved. Like, John, who’s been on this before, did his automations for... anyone can now like make automations or even build a small website and just try different things.

Julia: Yes.

Gabriel: And my hope is that we get even more kind of non-engineering, product design folks doing stuff.

Julia: Yeah, it’s so interesting because usually when we’re trying to automate something, we ask IT, like we, as in PeopleOps and folks who are not tech, that are in the non-tech teams, we usually need support to automate anything. And now we’re like able to automate things ourselves. And it’s really exciting. So we’re kind of like, are we, are we coding now? Yeah, I like it. Coding.

Gabriel: Yeah, yeah, you are.

Julia: And I mean, of course it’s all very basic stuff that we’re doing still as people who are not in tech, but it’s really exciting to see what you can accomplish with AI and what you can accomplish with just having that autonomy and freedom to just go and do it and do it yourself. And it’s all about curiosity, which is one of the things that I really love about hack days and the freedom that you get. It’s, you really, you can work on anything and you can explore anything and you can go in any team and be like, hey, I want to learn something about finance or the legal department or whatever. And just like actually be able to explore and get out. Even the folks who are in tech get out of the tech zone and kind of explore other things they can be impactful with. And so yeah, I love hack days. I think it’s super exciting. Like we, at some point we had less frequency and people had a very emotional response to it. So it’s clear to me that people really love it and that it has a huge impact. We have an engagement survey question about hack days and it’s usually pretty positive. So yeah, I’m very proud of the program. We do have some challenges like the growth and how we’re gonna manage the amount of demos and all of that, but I’m super happy to own it. And this was really exciting. Thank you. Thank you so much for having me here. It’s fun to talk about.

Gabriel: Yes, thank you for coming. I also love hack days and thank you everyone for listening. See you next time.

Julia: Thank you. Bye.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Peter: Hi, and welcome to DuckTales, where we go behind the scenes at DuckDuckGo and discuss the stories, the technology, and the people that help build privacy tools for everyone. In each episode, you’ll hear from DuckDuckGo employees about our vision, product updates, engineering, and more recently, approaches to AI as well. My name is Peter. I am on the product team at DuckDuckGo, often working on our browsers, which we’ll talk about today. And joined here today, I have my colleague Stephen.

Stephen: Steven, work on the product design team.

Peter: And Balint.

Balint: Hey, I’m on the Windows Developer Team.

Peter: Awesome, and today we’re going to talk a little bit about our browsers, as I mentioned. So many people don’t know we have browsers. They’ve come to know us through DuckDuckGo Private Search, which of course you can use in any browser. But we’ve for many years offered browsers on iOS and Android devices. And more recently, in the last few years, we’ve expanded that to Windows and Mac desktop computers as well. Our browsers are amongst the most popular in the market. For example, in the United States on iPhones, we are the most popular browser after Safari, which comes built in, and the Chrome browser, which of course many people have come to know over the years. So our browsers are used by lots of different people. When we started building our browsers and as we expanded our browsers to desktop computers, we started to prioritize the features and functionality in our browsers based on sort of two things. One, having privacy at their core. Of course, that’s our brand promise, and we want to deliver on that. But also, we wanted to make sure our browsers were easy to use replacements for the privacy invasive browsers that people have been using for many years. And so we prioritize a lot of features based on what would make it easier to use. We base it on people’s feedback, which we’ll talk about a little bit more. And often internally, we talk about focusing the features we build on the three Ds. We call that dependability, discoverability, and delight. The features should just work. It should be easy to find. And they should be delightful if we’re doing them right. So today, we’re going to talk about a couple different browser capabilities that fall into a few of these buckets that we thought would be interesting to share a little bit about. First, we’ll talk about custom theming. Stephen, do you want to tell us a little bit about that?

Stephen: Yeah, custom theming to me is pretty simple. It’s where you can go into your browser and just choose which color you want to use. It can be that green or blue or purple. I think we have a range of colors to pick from.

Peter: And why did we build custom theming?

Stephen: Originally, we didn’t plan to. We were going to try to keep it simple and just have a light and a dark mode. But we got into it we discovered that people really like to customize their UI. It’s just like a fun way to make browsing more fun and personalized. And they really like that. And it also can help you pick your browser out from other browsers. So it’s really also a functional improvement.

Peter: Do you want to give us a little bit of a demo on how that works?

Stephen: So we built two ways to get into this. If you’re on the new tab screen, you can open the customized sidebar and we have a theme picker up here with a few colors. You can pick gray or blue or green, purple. Lots of people love purple. And you can also change the theme from light to dark or just to use whatever the system decides. So we support all of these. And if you want, you can also change the new tab background screen to be separate from the rest. And we’ve spent quite a bit of time making sure that this works on all surfaces. So you can see it in the settings and the bookmarks in the history and elsewhere. Also, we have a neat little feature here where you can change the app icon and the dock to match the theme. So that’s a little more continuity and a little more fun added to the feature.

Peter: Stephen, were there any challenges in building this as we started to introduce it?

Stephen: Yeah, so like I said, we didn’t really have themes on the roadmap when we started. So we kind of had to go back and rethink the way we were styling things and build a whole theme system so that we could apply a bunch of different themes and make sure that it was, we could have more themes in the future. So now it’s all centralized and easy to work with. So we could probably add more or maybe even enable customization at some point, user customization.

Peter: And would you say, what’s the response been from our users so far?

Stephen: It’s been pretty positive so far. We got a lot of feedback saying, thank you for adding this. And it’s been pretty positive and monitoring people using it. It’s going up. So that’s good.

Peter: And you’re saying the certain colors were the most popular, I think you mentioned earlier.

Stephen: Yeah, so I think the most popular color is, let’s see, it’s sleet blue, I think. And then the second most popular color was violet. It’s kind of neck and neck there. Personally, desert is my favorite, but it was not the most popular.

Peter: Got it. Cool. Where are we going next with it?

Stephen: We’re looking into adding a dark mode for web pages that don’t have it, which is going to be pretty interesting. And like I said, we may, depending on if we get more feedback on how many themes we have, how many colors we ship with by default or expand that out to letting people choose their own colors.

Peter: Yeah, the dark mode for sites to use dark mode, that’s starting to roll out. We have that on mobile devices, on Android starting to roll out on iOS. And it’s something, as you said, we’re looking at on our desktop browser. So I think our users can look forward to that in the near future, which is awesome. Great. Let’s talk about password management next. Maybe Balint start with what is a password manager? Maybe people don’t realize that in their browsers they have typically password managers. Maybe just grab that at a high level first.

Balint: So the password manager is the thing that comes up when you go to a login screen and the browser offers you to fill your username and password automatically. And it’s a critical piece of feature because one of the most dangerous attack vectors used to be credential stuffing where an attacker got hold of your passwords from one site. And if you have been reusing the same password on other sites, they could just brute force. They could just try the same password with your username on other sites and usually get a hit. And the password manager is a perfect defense against this sort of attack because it takes the burden of having to memorize passwords off your back.

Peter: That makes sense. Yeah, I guess, you know, over the last 10 years, a lot of you hear a lot about third party password managers like 1Password or Bitwarden or LastPass and more people are using those. But I think it’s probably safe to say that most people use their browser as their password manager. Maybe they don’t even recognize that it is a password manager, but it is, as you’re describing overall. Why did we choose to sort of build it into the browser and start to build out a lot of those capabilities? Is it because that is the easiest way for people to protect themselves, as you’re describing?

Balint: I think it matches perfectly well with the 3Ds that we have. So it’s a core part of dependability. You want your online experience to just work, not have to memorize passwords and not have to think about where those passwords are stored. Of course, if you explicitly want to, you can use a password manager, but it is very nice to have a secure default built right into the browser. And we take care of the whole lifecycle. So when you register to a new site, our browsers will suggest secure passwords, which are randomly generated and long enough, take care of storing them, and if you turn on sync, can get your passwords synced across your different devices so you don’t have to re-enter them on every single device you have.

Peter: And we’ve had the core capabilities in our browser product for some time. What’s been the user response and maybe describe where we’re going with it next.

Balint: So we see on our anonymized dashboards that uptake for these password managers is very high and we keep adding new features. For example, recently over the past months, we have shipped a feature for Android and iOS devices where you can use DuckDuckGo’s password manager as the main system password manager and you can fill passwords in other applications from the DuckDuckGo database. And uptake for that feature has been phenomenal. We have almost half. So a bit more than 40% of our iOS users enabling us as their primary system-wide password manager. We have more cool features in the works. My personal favorite is something that is called TOTP. It is that six digit number you see on certain websites, which is used as a second factor authentication. This is something that many password managers already offer, but browsers don’t really tend to. We would be one of the first on the market to ship this. And it would present a second security layer that is much harder to phish, so much harder to steal than a conventional password.

Peter: And do you think we’ve hit challenges in building this along the way? And what are some of those challenges?

Balint: Certainly. So password management is all about security and figuring out a way that lets us store the information on the devices securely in a way that we can access them, but an attacker cannot access them. And we have been very proactive in using operating system level security features. For example, on Windows, we are storing them in a way that is secured by Windows Hello. So that is Windows’s own built-in encryption mechanism. So breaking our password manager is similar in difficulty to cracking Windows’s own encryption, which makes it a pretty hard target.

Peter: That makes sense. So when a user goes to access a stored password, that’s why the browser will cause Windows to trigger the Windows Hello authentication overall. Makes sense. And then you talked a lot about storing all these passwords. And, you know, as I mentioned earlier, a lot of people don’t know that we even offer browsers to start with. And so if they come to us and install our browser for the first time after many years of using another browser, presumably they’ll have a lot of passwords in that other browser that they need to get into our browser somehow. Maybe we can talk about that and how we’ve been iterating on the import capability, so a little bit.

Balint: Exactly. So we offer an import capability which you can do as part of onboarding. So when you first come to our browser, one of the onboarding steps is the offer to import your stuff, but you can also do it later on from main, from more tools and import on Windows. Similar logic is available on our other platforms as well. And one of the recent additions to this has been the so-called multi-import where you can very simply select not just one target or one source, but several of them, we can import directly from the most popular browsers on the market. And it’s been trimmed as well. So if everything goes well, it is just one or two clicks and you are done. We support importing passwords and also bookmarks. And this also sees pretty good uptake. So once we have updated the visuals for this multi-import, we have been seeing 13% more people choosing to import and over 70% more stuff getting imported, so 70% more passwords and bookmarks. And that is pretty good news because it reduces the friction. So when you come to DuckDuckGo, you don’t have to restart your digital life from scratch. Instead, you can continue from where you left off, except in a more private manner, of course.

Peter: That’s awesome. Yeah, my experience is, you know, over the years, people tend to often use more than one desktop browser, sometimes one for work, sometimes one for personal or for separation of different context. So the fact that we make it easy in one step to import from multiple at a time seems to be really beneficial and delightful to users overall. That’s great.

Balint: Yeah, and we are also innovating on our mobile platforms as well. So for example, we have been first to market with a feature that lets you import from mobile browsers directly. So not via a desktop browser that gets synced to your mobile version as well, but directly on the mobile browser itself. And we are planning to take care of the newly released iOS API, which will further streamline imports on the iOS platform.

Peter: That’s great. Yeah, we have that both on iOS where you can import from Safari and then we also have directly imported from the Google password manager on Android, which is great for people. Awesome. Well, thank you, Balint. Thank you, Stephen. I think as a summary, what I’ll say is if you are using our browsers or if you’re not and you choose to start to use them, we’d love to hear your feedback. We do very much listen to user feedback. There’s ways right in the product to send us feedback. You can fill out the form. We absolutely look at those, read those. If you go on our subreddit and discuss our browsers and capabilities there. We absolutely look at those threads and we take all of that as input to help make prioritization decisions about the features we’re building. So thank you very much and see you next time on DuckTales.

Balint: See you.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Gabriel: Hello, welcome back to Duck Tales. I’m Gabriel, founder of DuckDuckGo. I have with me today Zac Pappis.

Zac: Yep, I’m Zac. I’ve been here for about 14 years now, so I think we’ve had the pleasure of working together for a very long time. And we’re on our insights team, so generally doing a lot of market research or user research and generally trying to give us better insight into what our customers and future customers want.

Gabriel: Cool, and yeah, happy to have you. And we’ve been friends for 15 years. But we are going to talk about AI today. You’ve done a lot of research in AI over the last year. And we obviously recently put out this kind of Yes AI, No AI public poll. And we’ve talked about AI a few times on the show here, just in terms of product, but also just more generally. So our approach to AI has been a bit different than the other tech companies. Obviously we’re asking people if they want AI, no one seems to ever be doing that. But more generally our approach is to make AI features that are private, of course, because we’re a privacy company, useful, which I think lots of customers have different opinions on what AI is useful and what is not useful across all sorts of products. And most importantly, I think to this discussion, optional. So we’re making all of our AI features optional. You can turn them off or tune them, actually. We’re going to get into that too. So that’s been our approach, private, useful, optional. And we thought we would talk about, with you today, who’s been actually doing a lot of research with consumers and looking at other research, kind of how those three things kind of thread through the research. So if we were gonna, you know, obviously that’s a big topic, maybe we can start at the highest level and work our way down. Like, yeah, highest level landscape of AI right now, kind of like what are you seeing in all of the research trends that you’re looking at?

Zac: Yeah, I mean, generally, I think what we feel just as being consumers and people downstream of a lot of technology is that it’s been pushed on us as well as the general market, despite not really asking if we wanted it. And the market moves so fast that I think we feel the consumer has been left behind and no one really ever got an informed choice or consent into how prevalent AI has become in every product. And we think that’s a mistake, right? So for users and the companies that are building it, that choice as you laid it out is to make these things really truly useful and optional and of course, private. But that’s not what we’ve seen. And I don’t think that’s how most people have felt in terms of the ways that the companies have rolled these products into, or I should say rolled them out to their broader consumer base. Lots of tech companies have put AI overviews into products or turned AI on by default without really asking for consumer consent. It really reminds me a lot of what had happened through the 2010s and the cookie era or social logins or a lot of cases where technology just sort of appeared for people without a lot of demand for it or really a lot of cost to use it or a lack of concern into how exactly their data was going to be used or how would it impact the usefulness of the product they were using.

Gabriel: Yeah, that makes sense. I mean, giving them the best intentions, like, even though we kind of just screwed the approach, I think somebody saw a lot of people’s assumptions seem to be it was super useful and everyone’s going to love this. And I think to some degree it’s turned out not as useful for lots of people in different scenarios. And then some people just don’t want it shoved down their throat. You know, they would like the choice. They might use it anyway. So with that in mind, like, I know you actually — so like that’s kind of the general consumer sentiment, but like in terms of like actual adoption, like data points, where are you, what are you seeing right now?

Zac: Yeah, so we see that this is everywhere. We also see that people are using it a lot. Some studies that we have looked at and run ourselves, as well as those that we’ve evaluated from external companies and polling services, or things like Pew had run a study in June of last year showing that there is a lot of variability when it comes to who is using AI and who has a strong preference for it. A good example might be that 29% of parents use AI daily, but 15% of non-parents use it. And that’s almost twice the rate from parents and non-parents that are using it. That’s just the type of disparity, I think, that we see across different consumer types. So older or younger, more educated, less educated, there is a real preference and usage gap between these groups. And that doesn’t mean that this product is going to be right or the specific implementation is going to be right for everybody. And so that level of specificity or attenuation for exactly how the product appears to people is really important to us. So despite the fact that you have lots of parents, 30% of parents using AI daily, I think even more, 50% of Americans are actually concerned, more concerned than excited about AI. And that’s gone up over the last few years. So you have these two things happening. You have a lot more exposure and usage of AI and more familiarity with it. But as people are becoming more familiar with it, their concern is going up. And that to us is evident that someone’s really living with a choice that they didn’t really get to make.

Gabriel: Yeah. Well, I think one thing is usually said in there and it goes to some other data points I’ve seen very recently, correct me if I’m wrong, but there was another Pew that came out really recently on teenagers and it was about, I think about half the teenagers were kind of regular daily users. And then there was another one, just about ChatGPT usage and it was like maybe 40% of desktop people were weekly users. What I see on that is I do see that’s obviously very significant adoption in three or four years. However, that’s still like majority of people close are not daily users. You know, so like the one headline I take from what you said and what I’ve seen is that yes, there is a lot of adoption, but this idea that everyone’s using it all the time is that narrative just seems not true. And it relates to your concern points. Like the people who are using it are concerned. People who aren’t using it are also very concerned. And so there’s just like generally a building concern. There are obviously lots of different issues people have with AI. So like, how do you like think about that concern? If you kind of try to piece it apart, like, where are people expressing, how are people expressing that concern, I guess, and how is it related to their...

Zac: Yeah. I think it’s something that we know pretty well because we’ve seen a similar type of concern with tech overreach before. Just some stats that we have on hand to share because I did look into this a little bit before today. There was a study by Resilience [CHECK] in December, so pretty recent, December 2025, showing 54% of those polled, US adults that they surveyed, have avoided AI-powered features. And in our own polling of US adults, we’ve seen something closer to like 13% of people actively disabling AI search or browser features in their browser to protect themselves. And when we look more deeply at why people are doing that, taking, you know, kind of these extra steps either to tune or to completely avoid the way that AI has been integrated into their products, we see a couple of, I should say, familiar concerns. There is a concern that companies are rolling these new technologies into their products so quickly that they come with new types of privacy trade-offs or data security concerns. And of course, we’ve seen that in the past with Cambridge Analytica and other cases where a lot of data collection can just increase the surface area, the risk surface area for having that data. So when asked why people were avoiding AI or were turning these features off, I think 51% had said they reduced data sharing because of AI, meaning a behavior that they’re taking to proactively not share as much as a result of AI being in the product. And when asked what they wanted, the majority of answers from those folks were opt-out rights, data traceability, and disclosure. None of those things are no AI. They’re consistent more with a theme that would be control, right? Not no AI. They just want to know when it’s being used, how it’s being used, and to have some input and flexibility into where it’s applied to the product.

Gabriel: Yeah, I mean, those seem like totally legitimate concerns to us. I think more broadly, like what I’m hearing is, I mean, that’s, privacy is one of the main concerns people have. That’s obviously why we’re building private AI and giving people that control. The second is that people do want options to your point. It’s not just yes or no. It’s, like, yes, some people, but it’s a smaller percentage like you pointed out, totally want to get rid of all AI because maybe they have more objections for various reasons. But it seems like the majority of people actually just want it to be useful and private. So what’s useful to somebody may not be useful to another person. And so if there are 10 AI features, maybe they want to engage with six out of 10. Maybe they want to turn the dials on them a little bit different. And so it’s like this broader customization of AI thing to make it useful that we’re trying to do with our search features that I don’t think other companies have approached in the same way. They’re just kind of like all on all the time, you know?

Zac: Yeah, exactly. Funny enough, there was an Ernst & Young study pretty recently, it might have been January or February of this year. It was a poll of 500 or so US business leaders, so people in an executive position or director position in some kind of a large company. I think these were all companies of over 40,000 employees or something pretty large. And from that survey, 78% of the company leaders polled said that their adoption is outpacing their ability to do good risk management. And 45% of the same people polled said they had a confirmed or suspected data leak via these unauthorized AI tools. So they’re really prioritizing speed over the exhaustive vetting that they would need to do to either ensure that they’re actually producing something that’s safe, both from a privacy and security standpoint, and also useful, that they’re getting it into a product really without understanding user needs or how the product is being adopted by those who are really core to their business.

Gabriel: Yeah, that’s actually a really good point. I mean, it’s like we expect these numbers that we’re citing to change over time, right? And you’re already seeing that like concern could go down if you address people’s needs for transparency and control and also concern could go down as people discover, you know, actually useful features and how they’re using it. I think part of the issue is there hasn’t been that control and transparency. And then part of the issue is I think to that last point is things are intentionally moving too fast for people and change, you know, it takes time to understand these technologies, to get good risk management, to like get good processes in place that, you know, don’t exploit your data and have other security and privacy risks. And so, like, I imagine that longer term, some of this will settle down, but it feels, I guess, probably not the level, it just — if I were to summarize some of this, it just feels like it’s moving too fast for people a bit and that everything needs to slow down a little bit.

Zac: Yeah, I mean, it’s just kind of spitballing here, but it seems almost like a double-edged sword. You both have people who are in charge of making these types of product decisions, rushing them out the door without fully understanding them. And then as consumers who are also, you know, privy to, or I guess experiencing the downstream effect of other product changes, we feel it. So you’re getting this double whammy of having to participate if you’re somebody that works in any industry right now, which probably a lot of them are impacted by AI. It’s likely that your organization is dealing with a lot of these same challenges, not really having the right oversight or internal expertise to understand the risks. A lot of pressure from the market and competitors moving so quickly and you feel like you’re going to get left behind. And it’s understandable why some people might feel like the industry or the changes are moving faster than they can really make sense of them. And I think that’s what we feel both as people somewhat responsible for creating consumer technology, but also as consumers ourselves. We see them in the products that we use every day from Apple and Microsoft, et cetera.

Gabriel: Yeah, well, let’s take a few minutes to just talk about our Yes AI, No AI campaign we did. So we put out this essentially public poll. It really wasn’t to our users, and I’ll get to that in a second, about asking, are you Yes AI or No AI? And we understand, we just talked about how it’s all nuanced and it’s about control. So we understand it’s a bit of a binary kind of thing that we’re asking people to choose, but it was kind of like, your finger in the air, just kind of say what side you’re on. Now part of that was because we think that the people who are concerned inside just haven’t had a lot of voice. They haven’t really been listened to. So this was a bit of an attempt to allow and show that and have an opportunity for a tech company to kind of listen and see what’s out there. Obviously the poll ended up very skewed. Interesting though, is like overall numbers were like 85-15, something like that. But if you look when you polled our actual users on like a platform, it was more like 50-50. And what happened was the whole poll went viral in the No AI community. And kind of my theory is, you know, not a lot of people are speaking to this community. We did. And so it went viral there when there’s everyone speaking to the Yes AI community. They don’t really have a reason to vote Yes AI, you know, but people really do want to express their No AI vote. So I thought it was interesting. We didn’t really know what to expect. That was the hypothesis. And that’s really what happened. You know, it seemed like, I guess, I guess my read on it and tell me what you think is like, I think some people are out there and maybe listening to this and being like, I still don’t believe there’s no AI sentiment here. And it’s like, I think we’re here to say, yes, there is, you know, like we have so many data points here that show there’s a large percent of people who are concerned with different AI things.

Zac: Yeah, exactly. The campaign was awesome. And I know it’s not something that we typically do. So it’s great to see just a response from it. But it was really meant, I think, to point out, correct me if I’m wrong, the gap that we didn’t, that no one really gave consumers a choice for. So this gap between yes and no, where most people aren’t in this absolutist camp. Even if I think you have some anti-AI sentiments, it’s more than likely, I guess, just given a kind of a bell curve that for most people, they fall somewhere in the middle. It can be useful for some things, you know, in certain conditions or if, you know, it’s kind of an opt-in or something that really is explicit for the user. But in other cases, maybe not. And I think the experience that this campaign had kind of really drawn on was what we talked about earlier, just a flood of AI without really a lot of ramp up and consumers not really getting a choice to speak out about it. And that certainly, as you pointed out, the people who were really pro AI were kind of getting the life that they wanted to live in and the world was really bending in their direction. But the No AI crowd probably didn’t see or kind of people on the other parts of that bell curve didn’t really see anything coming for them. They felt probably like it was and it’s still seemingly like an AI-powered world that we’re heading into. But without understanding what that is, it’s certainly scary. And certainly with the other concerns that we’ve seen in the data that we’ve just shared here today, there are privacy concerns. There are systemic concerns and how that’s going to impact the rest of the products that they use. So if it’s something that’s getting built into Amazon, how does it impact my Echo device? How does this data migrate from one process or one product to another? And all of that, I think, just comes in tenfold with AI because it is such a sensitive topic for people. And the type of content that you engage with in AI is uniquely different from, say, something that you would type into your browser. It’s a lot more personal, it’s a lot deeper, and certainly from the history that has grown from that, it can be too personal for people and really jeopardize a lot of the concerns, bring a lot of concerns forward that they had in the past with cookies and just general corporate tracking. So one thing that I think ours does really well that we’ve seen a lot of positive response for is the fact that you can use it with no accounts. You can just go to duck.ai and start using it. No accounts, it’s not training on your data, you can stop using it when you want, you can turn it off if you’d like, and all of that optionality and that level of control is just not something I think we have seen in any other product. So it would be interesting to see.

Gabriel: Yeah, agreed. And people have asked specifically on the back of that campaign, kind of what are we doing? And it’s important to say, well, first of all, all that optionality was built in already before we did that. So you could have turned Duck AI off completely from search. You can turn Search Assist off within our search results on Duck AI itself. You can choose what model provider you want. Like you don’t have an account, like you said. But I’d say additionally, we also created this domain, noai.duckduckgo.com, as well as yesai.duckduckgo.com. And those have now built in like all AI on and all AI off. If you really want to be on each end of the extreme and you don’t want to tune it, we took the time to kind of like make those two bespoke experiences for people on the end. And we have seen a decent uptake on the No AI side. Just thinking about, you know, closing out a little bit, like, is there anything on the product side or going forward, you know, in addition to all that you want to talk about from like a research perspective or anything.

Zac: Yeah, I could talk for hours. I don’t know if the podcast can sustain that but yeah, it’s just something that you just said was interesting because I did run into my neighbor yesterday who is kind of aware that I work at DuckDuckGo and was asking me about the AI, you know, just in general, not ours, just in general, what’s going on with AI and they were really delighted to hear about noai.duckduckgo.com and I directed them to our search overviews and let them know how you can tune kind of the frequency that those appear in search. And she was like really taken aback by that. I think that’s the kind of experience that we’re trying to manufacture more of is that like the need clearly matches the product that we’re providing. And that connection happens almost instantly where people recognize that that’s what they’ve been looking for. So for us, I think that’s a lot of trust and control and really turning that into what you would call like a delightful UX to getting back to, I think, the podcast as well. So not really AI that confuses you, AI that should be there when it’s truly helpful and how and where that gets embedded into the product is a lot of what our research is focused on going forward. So you’re going to be seeing a lot of how we integrate and make AI easy to get to, easy to get out of, and easy to switch from when you’re navigating between, say, traditional search, browsing, AI. If your phone is in your pocket, your laptop is in your bag, and all of those contexts where a technology can be helpful, but you may not know exactly which one can be. So we want it to be present when it is helpful and kind of hidden or tucked away when you would like to invoke it, but otherwise out of the way.

Gabriel: Cool, that’s a good place to end. Well, thank you, Zac, for coming on.

Zac: Thanks for having me. This was great. Thank you. Bye.

Gabriel: Cool, thanks everybody for listening. See you next time. Bye.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Gabriel: Hello, welcome back to Duck Tales. I haven’t been here in a while. And I am Gabriel Weinberg, the founder of DuckDuckGo. And I have with me someone who I don’t think has been on Duck Tales at all yet, but you should know, Caine Tighe, who I know very well, who’s the first employee of DuckDuckGo and now our CTO. Caine.

Caine: Hi Gabe.

Gabriel: We’ve been working together for a very long time. And we’re here today to talk about something we’ve both been working on. Caine more than me, but I’m working on it some, which is our web search index. So as some background, first some background. DuckDuckGo started as a search engine, as many people know, and it was actually started by me. I was by myself for a few years. And the first thing I did was start crawling the web and building a web index.

Caine: Yeah, for sure.

Gabriel: But you know, I soon realized that that is very expensive, especially as one person. And there were other places to get a web index at the time. And what was more interesting was maybe adding value on top of the web index. So building other indexes, this was a time, this is the mid 2000s, you know, there weren’t, there obviously wasn’t AI, but there wasn’t even really many instant answers on search engines.

Caine: I mean, that’s what we were working on together at the very, very beginning. Like we were working on, you know, you had the knowledge graph. It wasn’t called a knowledge graph at a time, but you were doing all the structured content from Wikipedia and otherwise. We worked on some other smaller indices. So yeah. And then actually fun fact, in hiring our backend project is still based on some of the original spam and content farm crawling, like one of the projects is based on some of the spam and content farm crawlers that you originally wrote. So that lives on 15 years.

Gabriel: Yeah. So we were doing lots of indexing and lots of crawling. Yeah, exactly. Just not, you know, we started, but then we stopped doing a full web index, but just as examples, right? We started like the code that you were talking about indexing Wikipedia, which became our knowledge graph, you know, which is, powers a lot of answers, which also we used when we started working on AI answers. We’ve been doing local indexing for, you know, over a decade, local businesses and things like that. You know, then all sorts of kind of niche indexes that involve some crawling like lyrics and things like that. So indexing technology is not new to us, despite what some people say about it. Sometimes we do lots of search indexing, but we hadn’t been doing a full web index until relatively recently, last few years-ish. But now we are. And so the question is, the questions and why you’re here, and we’ll talk about it for a few minutes, is kind of why, what’s going on, how, all the main questions, which we’re obviously not all gonna answer today, but we can start with, I think, kind of the why, but why are we well positioned to work on this? So to speak, and you’re kind of at the center of it, so I think you’re a good person to ask.

Caine: Yeah. I mean, I think, the why now is a mixture of like our needs. Like we want to support our own AI use cases. That’s we have two primary agent, agent driven products out. Search Assist, which is on the SERP, search engine results page, duckduckgo.com. And then we have Duck AI, which is our chatbot. And both of those products are hungry for this kind of data. So it, yeah, it just makes sense for that.

Gabriel: Yeah, in particular, right. You could maybe talk percentages, but like there’s some percentage of search results now, what is it like 25%, I think that have Search Assist answers. And then, you know, the percentage better made for Duck AI, but some significant percent call the web, 15% maybe.

Caine: Yeah, I always do. I do, I have my numbers based on, absolutes make more sense. Yeah, yeah, just, yeah.

Gabriel: You know what? Bad question. Ignore numbers. Doesn’t matter. Good percentage of queries and Duck AI prompts require web search. And so we need a web index for it, essentially, right?

Caine: Yeah. I mean, I think on the chatbot side, it’s really good, like to ground. If you’re deciding whether or not to ground and you’re on the line, you should probably use RAG, retrieval augmented grounding, and go out to a third party data set. For us, raising the standard of trust online. We want to do that because the more that you ground, it’s known empirically, the answers are better. So we err on the side of grounding where I think maybe not everybody does. So it’s really good that we need to build our own index in order to be able to accommodate that. So that’s kind of some of the why now. And again, it’s on Search Assist and it’s on Duck AI. One of my favorite parts about this whole thing is like, we’re very used to working for customers, like our end user. For the search index, duckduckgo.com itself is the customer. A very nuanced, unique thing for us to be able to serve ourselves, which creates this really tight feedback loop internally. So it’s been cool to like use our own and we are live for, you know, some amount of the traffic today. That’s just growing day over day for these use cases.

Gabriel: That’s a good point because like, I think in terms of like our position, well, positioned to do it is, you know, being live, you know, maybe we talk about that a little bit, but like that creates a feedback loop that we have that a lot of people don’t have because we have, you know, many, many millions of people using our search engine and now Duck AI. And so we’re getting constant feedback about the relevancy of the search results that we’re serving, not to mention the fact that we have almost 20 years of evaluating relevancy ourself on our own search engine.

Caine: Exactly. Yeah, exactly. So like, humans are unsurprisingly and appropriately more, more critical of results than agents are. So it kind of creates this higher fidelity feedback loop because, you know, through our, through anonymization and whatever else, like we can privately understand what is most relevant on the internet for customers and users. And that really helps us to, positions us to be pretty competitive in the space quickly. So like, I think that’s kind of interesting and it’s exciting and like the true DuckDuckGo way as you and I know well, like we like to ship stuff. And so it’s been really cool to, yeah, like it’s just been really cool to be using it already and in production, our own index. And it creates that flywheel and we could, you know, use buzzwords like reinforcement learning and this, that, and the other thing. But at the end of the day, it’s just really the relationship of consuming your own internal API product. That’s the flywheel and allows you to like establish relative priority really quickly and be like, I ran this experiment. Like we really think this query set is going to be well suited to our own index. And then we tried and we’re like, we’re not working that well on that. Let’s move to this other one. And then it just changes the game for how quickly you can iterate, which has been really exciting for, and I know the team’s really excited about it too, because engineers like to ship things. So that’s been cool.

Gabriel: Perfect, I think that’s a good intro. But let’s do, to your point about buzzwords, let’s do a few more buzzwords in terms of like, just give us the broad tech flow, like, you know, without getting too deep into anything, but just to give people a sense of kind of how it works and then maybe.

Caine: Yeah, so kind of the way that I think about this is like a little pipe or a train or whatever. You have your frontier that kind of is the web that you’re looking to crawl, like, because you have to pick what your frontier is. Then you crawl that. Crawling, all of these components are extremely complicated by themselves. To crawl, it means like, you need to crawl politely. Some sites want you to crawl, some sites don’t want you to crawl. And so to be a good trustworthy netizen, you have to respect those things. And that’s an important part of crawling. It’s also important to have the bandwidth and the throughput to crawl at the scale that you need to crawl. And so fortunately for us, we’ve had a lot of experience with that, so we have that. The rendering side, you have to, when you fetch content, you have to render, including JavaScript and everything else like that. The only way to get the content is to literally run the whole webpage. Otherwise, like you get no content. So that’s quite an expensive process. So we kind of do a naive approach and then a more complicated rendered approach. Then you have content extraction, which is like the next step, where you think about your title, your description, your headings, metadata, main body stuff, where you extract the content, what the page actually means. And then we’re very fortunate in today’s day and age to have semantics. So semantic search is a big part of the pipeline. And what that means is what people are calling embeddings. And you calculate embeddings on extracted content. And then we use a database, which I quite like, called Vespa. And it’s all ingested into Vespa. In my opinion, kind of your indexing, your ingestion, your features, how you calculate those things, and how they describe, that’s a big description of your product. Because it limits what you can do in the ranking phases, which we could get into. I don’t know if you want to get into the ranking funnel as well or just the data pipeline.

Gabriel: I think that’s a pretty good overview. I mean, I think, we’ll do more of these in the future, but like, if people have questions, they can write me or write us, but we need to get an email going where people, we have the proper feedback for this podcast.

Caine: A mini series as it were. Yeah, because I’d like to bring in, you know, I can give you the overview, but there’s a lot of experts on the team that, yeah.

Gabriel: Yeah, so I think what we plan in general to come back and we can talk deeper about specific things in the future, but I think that was a great overview. I think we hit it. Anything that we missed that you want to cover or we’re good?

Caine: No, I mean, I think the space is quite interesting and it’s been really fun. Like when we started out, you know, I started out with you 15 years ago and like, we were doing something different back then. The concept of a customer being an AI agent was not really a thing. And I think AI agents as search customers is quite interesting because like, they generate the queries. AI agents are generating the queries. They’re there. The queries are formulated quite well. They don’t feature misspellings. AI agents can consume large amounts of information quickly. Like lots and lots of information. They can speak in embeddings. So I just think it’s a really exciting time. And I think we made the right call to kind of forgo it until more recently. Because now we’re quite well equipped to support ourselves. And we have the products out that really express the needs and the requirements of an API like this. So I know I’m very excited for that, and the team’s very excited for that. So yeah, we should do more of these and get deeper into what a polite crawler really means, as opposed to just glossing over it, because it’s important.

Gabriel: Cool, well, thank you, Caine, for joining, of course. We’ll do more again later. If anyone’s got questions, email them in to us. Otherwise, see you till next time. Bye, everybody.

Caine: Later. Thanks.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Mary: Hi everyone and welcome to Duck Tales where we go behind the scenes at DuckDuckGo and discuss stories, technology and people that help build privacy tools for everyone. In each episode you’ll hear from employees about our vision, product updates, engineering and our approach to AI. I’m Mary McGee, I work on brand and marketing here at DuckDuckGo and I have two fellow product folks here with me today that I will let introduce themselves. Beah, why don’t you start?

Beah: I’m Beah, I’m on the product team. How was that?

Mary: I mean, I couldn’t have done that. You did great. Bobby.

Bobby: This is stuff. I’m Bobby. I’m also on the product team.

Mary: Wow, this was good.

Beah: And furthermore, we both have the word burger in our last names.

Mary: That’s true.

Bobby: This is true.

Mary: We are here to talk about browser onboarding today. And we have both Beah and Bobby here as folks that have worked on onboarding over the years. We can take you back to its origin. We can take you to some of the new changes. That’s kind of what we’re gonna do here and talk a little bit more about it. So why don’t we jump right in? I think, you know, why don’t I ask this to you Beah? Why don’t we start with you? What do we mean by browser onboarding? Like what is effective onboarding and why do you think it’s important for a privacy browser to nail this?

Beah: Yeah, so I think of onboarding as like, basically, the user’s first experience getting to know the app, what is this, how do I use it, and potentially beyond that first experience, could onboarding could last over multiple uses of the app or multiple days. But like the thing that you need to do to get a user from interested, sure, I’ll download it to like, I know how to use this. I know what this is. I know what the value is to me. I know that I want to keep using it ideally. And it’s really important, I think, for probably any app because many, I think for most app categories, the majority of users become non-users after one initial trial of the app, if not the majority, certainly a lot. And so that is a very important moment to show users what you are and what value you can bring to them. And to your question about for a privacy app or privacy browser, I think one of the challenges is that our apps are browsers with a built-in search engine, and all of that should just work. And the privacy protections that we’re providing, for the most part, aren’t really visible for intentional and just organic reasons, like we don’t want to get in your way, and like when, like the absence of bad things is just kind of invisible. So like, you know, I think it’s important for us in particular to like communicate in onboarding what actually is happening when you use the app.

Mary: So there’s like an explanation component of like how to use the app. And then there’s sort of a first impression, like this is who we are, this is how you should feel using it. Bobby, like can you talk a little bit about the role of the more like emotive kind of relationship in onboarding?

Bobby: Yeah, ultimately, there are, I think, stages to actually understanding and feeling connected with an app. And the first one is just knowing what it does. But there are stages beyond that which are feeling connected with what it does and really feeling like you understand it through experience, not just in theory. And so we try to reveal both through the tone with which we describe things and just really emphasizing demonstration and nudging you to try our features and see them and experiencing them for the first time and let you draw your own conclusions about whether that serves you.

Mary: There’s this line in our current onboarding that’s like, it’s something about a tracker losing its wings. Beah, has that been in there since the beginning or is that like...

Beah: Well, yeah, since I’ve been working on onboarding, which is like seven years maybe now, it was one of the first big projects that I worked on after I joined DuckDuckGo. Yeah, I think that, I mean, that copy was part of the, I think every time you browse with me, a creepy ad loses its wings. I think that copy came out of the very first version that we pushed live. And yeah, I don’t know. I think it actually hits with people. I mean, maybe, probably not all people, but just all things. I just remember, first of all, the early versions of that onboarding, some of the copy was super silly. I was hand sketching things and just saying whatever I wanted. And then our copy team had a little conversation with me and was like, hmm, maybe we can dial it down a smidge and I was like okay yeah maybe.

Mary: It’s, yeah, but it’s a good example of what Bobby you’re mentioning and Beah, your point. Like, it’s like, how do you talk about what the product does, but do it in a way that develops that sort of emotional connection. And it’s a funny testament that it stayed around this long. Since you, go ahead.

Beah: I was gonna say, I remember like user testing in like doing video user testing sessions with some of the early prototypes, which were like messy and weird and like hearing participants like laugh out loud, like chortle, say like wow and shriek or like, get mad. Like just have like an emotional reaction that like I definitely had not seen in user testing other components of the app. And I remember being like, I was still relatively new to DuckDuckGo. And I was like, this feels like it’s like doing something different, but maybe I’m just like, want it to feel like that. And I remember sending the videos to our like, kind of head of user insights and being like, am I wrong? Or is this like, are people like really having a reaction? And he was like, no, that’s yeah, they’re spitting their water out. Like, yeah, that’s interesting.

Mary: A chortle. Haven’t heard one of those in many, many years. I mean, since you mentioned it, Beah, like what was onboarding before? So we were talking about onboarding as this like personality, emotive. We can get into who Dax is as we talk about this, but I’m curious, like when you started, when you were doing these explorations and these videos, like what did we have and what were you, what sort of made you start doing this? Like why did you start the exploration that you did?

Beah: Yeah, it’s funny because we had these, I think the onboarding that we replaced was these tooltips that popped up as you moved around the app that were like, this is the fire button, it burns things, this is where you type, I don’t know. And I realized in retrospect after we launched this that on paper it was not that different. We went from tooltips explaining some of the features of the app to what we call Dax dialogues, our mascot Dax telling you things, but there were just a lot of details, I think, that made it feel very different. You could almost imagine the same onboarding coming out of the same written spec, like both, sorry, both very different onboardings coming out of the same written spec, but in practice and application, they felt very different.

Mary: Hmm, that makes sense. Do you have, I know you have maybe some images to share at some point, but yeah, we can always throw these in in post. But I want to, if there’s, yeah, if you want to share, we can always take a look.

Beah: Yeah, yeah, yeah, I can share things. I don’t know. Wait, okay, I think I can share things. Oh boy, I gotta do settings. You talk amongst yourselves.

Mary: In the meantime, Bobby, I’m curious, like you came into onboarding. So Bobby came in and was sort of tasked. It was maybe your first project of how can we improve this thing that is working really well, which is not the easiest task. I’m curious, like, what was your first impression? What did you think was working? Where did you want to focus your time?

Bobby: Yeah, I think, well, we’ve been talking about the feeling that we were trying to evoke from people. And one thing that immediately was already working in the version that Beah had previously done is helping you feel confident in the app about how to protect your privacy. And I think that’s a pretty high bar. That’s kind of a challenging thing. Privacy protections can be this vague, intimidating concept, but the tone and particularly using Dax’s voice to deliver these messages was not only unique to really any browser that I had seen, but certainly unique for a pretty serious topic and something that is somewhat technical to understand. So the first thing that stood out to me was that lighthearted approach and the lighthearted language while still being very clear that you can be confident that your protections are active and working for you. And then I guess the second thing I mentioned a little bit earlier, which is just really focusing on demonstrating and helping you experience things. The aha moment for me, and I think a lot of the user tests that we observed was when you see the trackers that are being blocked on the first page you visit and it names the companies. You can find that on any page in DuckDuckGo just by clicking on the shield in the address bar, but to reveal it and the first time you visit any page with trackers that are blocked, you see that list right at the top of the screen. And I think that is really illuminating and eye-opening to understand that it’s actually working and that these trackers might be even more prevalent than you think. And they’re probably popping up on nearly every site you visit, which I think is a good, both a helpful way to understand privacy protection in general and maybe a shift toward privacy browsers are actually good everyday browsers.

Mary: And what were you trying to improve? Obviously you felt like that connection, that demonstration component was there. What were you hoping as you started watching these user testing videos? What did you want to change?

Bobby: Yeah, there were a few things. I mean, we wanted to build on that really, build on using those tools, using the tone and using the demonstration mechanisms to one, highlight more features. So search wasn’t actually highlighted at all during the onboarding flow and we know users love our search. And so we wanted to help people experience that for the first time, notice that it has fewer ads than other search options. Second, we wanted to increase the trial of those features that we were highlighting. So there, that is a drop off, a potential drop off moment where people are like, I downloaded this, but I don’t actually have anything to do with it immediately. I didn’t come because I have one search in mind or one page I’m trying to visit. So they open it. They’re like, okay, I kind of get it. And then they close it and maybe forget about it. How can we actually help you just get through that flow, see how the features work the first time. And then also adding some concrete benefits when we’re asking you to set it up a little bit. So I can elaborate on this a little bit, but we really wanted to introduce the concept of this is a browser. You can directly compare it to your existing browser, Chrome or Safari, wherever you’re coming from. Almost everyone is starting with an existing browser. And so we want it to be familiar, but differentiated. And when we ask you to set those things up, how can we help you set it up as very familiar to what you’re already used to?

Mary: It’s such a good point because we’ve even found, you know, like we get social comments, we get feedback from users. There’s just a lot of skepticism that it can work just as well because, you know, people have been told that you have to sacrifice your privacy for convenience, you know, for the tool, you know, the bookmarking, the sites to work as you want. And so there’s just a lot of built-in, well, it can’t work, it can’t work the same. I can’t use it the same. And so what you’re talking about is like that direct comparison. It really, it seems like it really addresses that straight on. So I’m not surprised that that was effective. Beah, do you have anything to take us back?

Beah: Yeah, yeah. You were talking about the like, aha moment of seeing that we block trackers. Let me show you guys the, maybe, probably the first, this is like just a prototype in Figma, a design tool that we made. So that moment here, and by we made, I think I mean I made it, the design finesse is really something. So yes, this is the first version of that, Uncreepify it. Those arrows were just like, I don’t know, I need people to look at this URL bar somehow.

Mary: Oh my God, look at the old Dax. Look at his skinny beak.

Beah: Privacy grade. Yeah, yeah.

Mary: Wow. That’s pretty good.

Bobby: I’m sad I never got to experience the bent beak. It was already smoothed out and straightened out by the time I got here.

Mary: Indeed, indeed. That’s really cool to see because even though obviously, you know, we don’t have the privacy grade anymore, Dax looks different. The structure of this has really maintained, which was just pretty cool to see.

Beah: Yeah, I remember by the way to the point about like people have to understand this is a replacement browser and that it is in fact like good enough to... I remember one user testing fella who was like, you’d be a fool not to use this. That was like his takeaway. I think he used those words like, and I was just like, yes, like that is like, right, right? Why wouldn’t you want your browser to block this stuff? But yeah, it’s like he had to like get a certain amount of engagement with that, with what we’re trying to do in onboarding to really process and appreciate that and believe it.

Mary: That makes sense. I mean, to your point from the beginning, Beah, it’s like much of the product benefits of DuckDuckGo are the absence of things. It’s like the absence of ads or trackers or cookie pop-ups. That’s actually really hard to communicate because you’re not usually switching and using both and noticing that difference. Is there anything that either of you remember working on this that was like particularly tricky to figure out how to explain or make emotive or find a way to communicate from that perspective?

Bobby: I actually think one of the trickiest things, this isn’t related to those privacy protections, but one of the trickiest things is helping people understand how a browser fits with their OS basically and how to reconfigure their settings. Basically, one of the first things we ask you to do in our app is leave our app and go change your OS settings. That’s a tricky thing to explain that we want you to leave, but we also want you to come right back so that you can understand everything else. That’s a tricky moment. But I think more broadly in terms of privacy, we tried to introduce some consistency to help you feel that your privacy protections are active at all times. And so we use language like protections activated is on the second screen. And then we now have consistent usage of the screen shield that shows up in the address bar on our app, on our search results page, in our Duck AI products, all of which is supposed to just remind you and reassure you and help you feel confident that the protections are actually working for you. And we try to just show you one time what happens if you click on that. It shows you the details of what exactly is happening, what is being blocked, what is being prevented. And I’m reasonably confident that just that one use or an occasional tapping on those details helps you recognize that every time you see this green shield from now forward, behind there are those details, but you don’t need to worry about them most of the time. So highlighting that I think can be a really effective way to minimize distractions in your everyday usage, but introduce it with a reinforcing mechanism as you do.

Mary: How do you think about the cost of onboarding? Like, you know, someone has downloaded this app. I know you said, Bobby, that people won’t always have one search they have in mind, but they’ve downloaded it to probably do something. And we’re, you know, when you create an onboarding experience, you’re saying, before you can do that thing, you have to do this. Like, how do you think of that trade-off of like the user intent versus like what we want them to know and appreciate about the product that they’ve downloaded? For either of you.

Beah: Yeah, I mean, we struggle with this quite a bit and like, and there we get feedback that people don’t want to go through onboarding and we’re sympathetic like, you know, most internal employees who work on the app have gone through onboarding approximately 7,583,000 times like, I get it. And in fact, we’ve experimented with giving people a big skip option. And even when it’s just an option, and we don’t make it super, try not to make it super tempting, but try to let people really self-select in. It has actually harmed retention as we’ve been able to measure it, which is sometimes a tough metric to move. So yeah, it’s tricky. We try really hard to just, I think, pick the content that is most relevant and also make it easy to dismiss and get out of the way if you’re halfway through and you’re done and not having it. And Bobby’s done a lot of work on that, like just making it more dismissible.

Bobby: Yeah, we, I mean, we do, every screen is dismissible. So if you’re just not, if you’re trying to get to something right now, you can just click the X at every point and do whatever you’re trying to get done. And then we’ll still introduce some contextual nudges and tips as your path unfolds. But one other thing to mention about the efficiency versus like the efficiency of getting through onboarding versus helping people really understand and learn the benefits. That is in user testing, the most common challenge in really understanding are people getting through this or are they just annoyed by it? I really believe that Dax’s tone gives us a little bit of leeway and offsets some of the frustration because you can literally, as Beah mentioned, you can hear people laughing and smiling as they go through something. I think that automatically offsets a little bit of the, am I doing this? Because at least you’re not having like being forced through what feels like a lecture, hopefully. That’s what we aspire to. I know some people still do feel that and they still feel like even in that tone, or especially in that tone, it’s kind of annoying. But ultimately, I think more user testers feel really good and smile and recognize that they’re having a little bit of fun on their day also.

Mary: Who could get mad at the duck? Come on. So cute. Yeah, terrible. All right. Before I wrap up, is there anything either of you want to say about onboarding that I didn’t ask you or anything you want to call out?

Beah: Maybe just riffing off of what Bobby was just saying, I think when we originally worked on this format, the thing in my mind was really a video game. I’m not a gamer, so I don’t know what the language is for the video games where you’re exploring a world and you get the dialogues. I think I’d been recently introduced to Animal Crossing, probably COVID times. So maybe there’s some inspiration in that.

Bobby: I’ll just say one more thing, which is that we’re doing even more. We’re still working on onboarding. We’re always trying to improve it. So we’ve got some new and exciting stuff coming soon that I think will make it even more illuminating and hopefully a bit more fun.

Mary: Yeah, and I can attest that Bobby reads all user feedback related to onboarding. So if you have some, please share. I can guarantee it’ll be read right over there. All right, closing rapid fire questions. I’ll ask you both just to. Beah, how long have you been at DuckDuckGo and why do you work here?

Beah: About seven years, or maybe coming up on eight, so a while. Why I work here, because I get to hang out with people like you and talk about fun things like duck onboarding.

Mary: Love it. Bobby, same question.

Bobby: I’ve been here a little over two years and I guess I work here because I used the product for a very long time and I liked contributing to something that I value. Particularly like I came probably for the unpersonalized results and I stick around to help more people discover them and improve them.

Mary: Hmm. Okay, that answer didn’t involve me at all, but fine. Yeah, yeah, I preferred Beah’s answer.

Beah: I felt upstaged like Bobby was like taking the high moral ground. I’m like, okay, it’s just rapid fire Bobby like, but cool.

Bobby: I was anticipating another question, but maybe not which is where I was gonna mention you, Mary, but personally, by name and how much I love working with you.

Mary: Nope. Nope. No, we’re cut for time. Cut for time.

Bobby: Alright, fine.

Beah: I work here because I like seeing Bobby dress up in Dax costumes.

Mary: Yeah, famous for it, always. Yeah.

Bobby: That’s a good one. Well, something that surprises me about working here is that not only are people very smart, but they’re also very funny. You two are very funny and there may be some other very funny employees, which keeps every day entertaining.

Beah: Thanks, Bobby.

Mary: Yeah. It’s actually, I said something similar, when I answered this question. So that’s interesting. All right. Thank you both. Thanks, everyone for watching. See you next time.

Bobby: Thank you.