[2026-11-13] eagle: work/zoom-transcripts/2026-10-20 Apple Weekly Team Meeting - Zoom ID 2026-10-20 20 01(GMT+6 00) gBBa-iiUQLycw55WRV1tnw/transcript.json work/zoom-transcripts/2026-10-20 Apple Weekly Team Meeting - Zoom ID 2026-10-20 20 01(GMT+6 00) gBBa-iiUQLycw55WRV1tnw/transcript.md work/zoom-transcripts/index.json work/zoom-transcripts/undated Alex Martemyanov's Zoom Meeting 3sv2crbiShun--0lAywwvg/transcript.json work/zoom-transcripts/undated Alex Martemyanov's Zoom Meeting 3sv2crbiShun--0lAywwvg/transcript.md work/zoom-transcripts/undated All-Company Meeting bPprGg5DTye9H-RQXkwMuw/transcript.json work/zoom-transcripts/undated All-Company Meeting bPprGg5DTye9H-RQXkwMuw/transcript.md work/zoom-transcripts/undated Apple Weekly Team Meeting - Zoom ID VSuzeqZCR5O2z5KtmaQt3g/transcript.json work/zoom-transcripts/undated Apple Weekly Team Meeting - Zoom ID VSuzeqZCR5O2z5KtmaQt3g/transcript.md work/zoom-transcripts/undated Costas Brammer's Personal Meeting Room 7JZgnceAT1aZ2H75ok4dmA/transcript.json

This commit is contained in:
Alexey Martemyanov committed 2026-11-13 11:12:48 +06:00
1 parent f209a2e692
commit 45adf056e8
19 files changed
+4388 -14

No files matched your search

@@ -19,6 +19,676 @@
"time": "20:05:36",
"speaker": "Jonathan Jackson",
"text": "Mm, got it."
},
{
"time": "20:05:37",
"speaker": "Tom Strba",
"text": "And I think I'm next week. Because I almost got a heart attack when you said that I am…"
},
{
"time": "20:05:40",
"speaker": "Dominik Kapusta",
"text": "Yes, this week is Diego. And sorry for not closing my task."
},
{
"time": "20:05:41",
"speaker": "Jonathan Jackson",
"text": "Okay."
},
{
"time": "20:05:45",
"speaker": "Tom Strba",
"text": "that I am this week."
},
{
"time": "20:05:45",
"speaker": "Jonathan Jackson",
"text": "This week… Why is Diego… I don't see a task… this is a macOS release we're talking about, right?"
},
{
"time": "20:05:52",
"speaker": "Tom Strba",
"text": "Yeah, I think Diego maybe marked it as complete in the beginning of the week, because there's just, like, triage…"
},
{
"time": "20:05:57",
"speaker": "Jonathan Jackson",
"text": "Yes, you're right, Tom, your block is the 26th to the 30th, because that task is missing, I didn't see it, I wasn't looking at the dates, I was just reading the list."
},
{
"time": "20:06:04",
"speaker": "Tom Strba",
"text": "That's okay. Just wanted to be sure."
},
{
"time": "20:06:07",
"speaker": "Jonathan Jackson",
"text": "Okay. Diego is responsible for the release this week. It sounds like he's actually doing that. Is that right?"
},
{
"time": "20:06:08",
"speaker": "Tom Strba",
"text": "Perfect."
},
{
"time": "20:06:14",
"speaker": "Jonathan Jackson",
"text": "He knows. Okay, cool. Thank you."
},
{
"time": "20:06:14",
"speaker": "Dominik Kapusta",
"text": "That's right."
},
{
"time": "20:06:18",
"speaker": "Jonathan Jackson",
"text": "Um, no problem. Okay. Anyway, think about starting your self reviews this week. If you feel like you're gonna need more than a day to knock it out. Bartek, over to you."
},
{
"time": "20:06:31",
"speaker": "Bartek Waresiak",
"text": "Yep, I will be stepping down as IS app DRI. Jacek will take the primary role, and Brindy will take on secondary role. So, yeah, if there is IS drama, if there are IS issues, if there are any urgent things. Go to them! And thank you very much."
},
{
"time": "20:06:57",
"speaker": "Jonathan Jackson",
"text": "All right. Thank you. An, you're next on the agenda."
},
{
"time": "20:07:08",
"speaker": "Anh Do",
"text": "Oh, that was quick. Yeah, so in OK, there have been a few projects that involve native changes required as part of the front-end-led projects. Um, so the background is that they, uh, want to contribute some PR, and, uh, um… AI assisted Pi, of course, and then what have been working for us lately is like depends on the size of the task like I would be the one to like either. Dri the change that they are they want to make, or like they prepare the Pi, and I do a thorough review of that. So and now we want to kind of do this more regularly. And so there was a need for some kind of like guidelines that we can give to contributor from outside the apple team. So I was just starting like a small task to basically collect ideas from people. And if you have anything that we already have somewhere, maybe I could get put in. And so. it becomes something that we can point other people to when they want to contribute to the Apple codebase to ensure our standard and make sure that, like, everything is being done the way that we want. So that's about it. Nice."
},
{
"time": "20:08:19",
"speaker": "Jonathan Jackson",
"text": "Cool, thank you. I just took a quick peek. I see you do mention the engineering expectations. I think that's a good source. We might also want to link… to the onboarding process. Not that they would need to follow all of it, but some of… that's basically our best source of, like, how to get set up with the codebase. So there's probably a little work to do to figure out how to have a slimmed down version of that that folks could follow. And I'm sure other people have other ideas as well. So thank you. Um… Luke, we have a non-Apple person lurking in our midst. Welcome. What can we do for you?"
},
{
"time": "20:08:54",
"speaker": "Luke Andrew",
"text": "Hello. Yes, thanks for having me. Luke from SRE, for people that I haven't met before. Um, this is just a plea for generic Apple help. I don't want to, sort of, get all of your attention to focus on this task. We basically have… a looping client that seems to be sending, uh, requests once a second to us, uh, that is Apple, but not DuckDuckGo Apple, based on the user agent, and doesn't seem to be Safari either. Um… And it's cumulatively across lots of different devices, lots of different queries, sending in the region of 3 million searches a day. Which is substantial, not sort of catastrophic, but substantial enough searches that it's something that we want to deal with. Umm. And I don't really know where to go next to try and understand where it might be coming from or how we might try and reproduce it. So I don't want to solve that in this call, but I just wanted to bring your attention to the task. And if there's anyone who thinks they might be able to help, I'd really appreciate. a second pair of eyes, and pointed out that it's not even worth, with what we have at the moment, that we can, like, escalate this to Apple, because we just don't have enough, uh, information to get any help from them. So yeah, that's it. If anyone has any questions immediately or async, just pop them on the task. Yeah, that's all I have."
},
{
"time": "20:10:18",
"speaker": "BA",
"text": "Quick one, Luke."
},
{
"time": "20:10:18",
"speaker": "Jonathan Jackson",
"text": "This is not from a single client, but from a particular app version, potentially, is that right?"
},
{
"time": "20:10:24",
"speaker": "Luke Andrew",
"text": "Yeah. Yeah."
},
{
"time": "20:10:24",
"speaker": "Jonathan Jackson",
"text": "Okay. You know, be scared that it was one client going crazy. Okay, um…"
},
{
"time": "20:10:30",
"speaker": "BA",
"text": "Yeah, I was going to ask, what's the — we eliminated bots from the equation. Are we happy that it's not bots?"
},
{
"time": "20:10:36",
"speaker": "Luke Andrew",
"text": "It doesn't look like bots. The queries all look human. The rate is really low compared with bots. The moment that I put in a block for it and it immediately stopped, like redirected away."
},
{
"time": "20:10:39",
"speaker": "BA",
"text": "Okay. Right."
},
{
"time": "20:10:52",
"speaker": "Luke Andrew",
"text": "It got, like, a 302 from deep, and then just immediately stopped, so that doesn't feel like… bossy behavior either. They normally persist a little bit and then give up. Whereas this was just immediately, it broke out of whatever loop it was in and stopped."
},
{
"time": "20:11:09",
"speaker": "Jonathan Jackson",
"text": "Interesting."
},
{
"time": "20:11:10",
"speaker": "Luke Andrew",
"text": "Uh, so don't all take a look, because that would be a waste of time, but if you think you can help, um, I'd appreciate a pair of eyes. I'm gonna drop off. Thank you all for having me."
},
{
"time": "20:11:19",
"speaker": "Jonathan Jackson",
"text": "Yeah, one last quick question. It sounds like you're treating this as something that definitely needs attention, but not an incident at this point. Is that correct?"
},
{
"time": "20:11:25",
"speaker": "Luke Andrew",
"text": "Yeah, that's correct. It's been going on for… Based on… um, how we categorize queries for, um, like, ads perspectives. These get classified as a separate category based on user agents, so we don't send them to Bing and say, these are definitely good queries. Um, it's been happening for about a month. So it's, at the moment, not urgent. I'm gonna raise it with SRE, maybe, and see if we think it should be an incident that we just let run, but at the moment, that's not how we'."
},
{
"time": "20:11:55",
"speaker": "Jonathan Jackson",
"text": "Okay. Great. Thank you for the clarity there. Uh, and yeah, if anybody thinks they can help Luke out and figure out what's going on there, feel free to hop into the task. Thanks for coming. Moving on. Alex, you're first in the regular topics list."
},
{
"time": "20:12:16",
"speaker": "Alex Martemyanov",
"text": "Yeah, hey. I have several topics for today, including the VS Code integration postmortem to run later. And the first one goes with a proposal. The proposal is pretty short. And kind of related to the, uh, recent incident with Mac OS 12 crash. that we are not. Testing for macOS 12 and 13, I guess, because we are missing the infrastructure on CI runners. And the suggestion is pretty simple to extract the macOS UI tests target from the main macOS Xcode project, leaving it as a separate macOS project inside the. uh, Xcode workspace. So it will be available the same way it is now, and, uh, runnable. So only, uh, the only issue I can see for those who use, uh, the actual macOS Xcode project instead of the workspace. they would have to switch to another project to run the tests. So, I just wanted to hear if there is any concerns or objectives against it."
},
{
"time": "20:13:39",
"speaker": "Dominik Kapusta",
"text": "I think some time ago, we… someone asked… who uses or doesn't use Workspace, and people who didn't use Workspace were in a… In a minority, anyway, like, very few of them, so… I guess… I could move to the workspace."
},
{
"time": "20:14:06",
"speaker": "Jonathan Jackson",
"text": "I think it'd be good to ask that question in that proposal, at least more explicitly, and just give people a chance to chime in if they feel like that would be a hurdle. Um… Other question for me is, is there… should we think about the iOS side of this as well, and to match… structure, or is this really a macOS only question?"
},
{
"time": "20:14:26",
"speaker": "Alex Martemyanov",
"text": "Okay. Maybe, I… Didn't think about it, actually. And the main concerns here is the compatibility. So we can reach macOS 12 compatibility, at least for some local runners that I'm going to present further. And the build times, so we have package fetching and building for every macOS runner that runs the UI tests and any of the dependencies. None of the dependencies are used during the runs, so it's a 10-second step for each runner that can be easily omitted, and so, yeah. And yeah, it's not related to the main browser targets, so there's no risks. Uh, and they're both, uh, U.S. Probably to think about it in the ER."
},
{
"time": "20:15:25",
"speaker": "Dominik Kapusta",
"text": "I think iOS is not…"
},
{
"time": "20:15:25",
"speaker": "Jonathan Jackson",
"text": "I mean, I don't have a strong… sorry, I don't have a strong feeling there, I just felt like if we're gonna make this change, like, if it's gonna end up with different structures depending on the app, we should think about whether we really want that to be the case, or if we should do the other."
},
{
"time": "20:15:37",
"speaker": "Alex Martemyanov",
"text": "No, it actually shouldn't change any structure, any file structure, just the project placement."
},
{
"time": "20:15:38",
"speaker": "Jonathan Jackson",
"text": "Screwed up. Yeah, yeah, yeah. Dominic, I interrupted you. I'm sorry."
},
{
"time": "20:15:44",
"speaker": "Dominik Kapusta",
"text": "Ah, no worries, I just wanted to say that iOS is not affected by this at all, because… iOS doesn't need testing on various macOS versions."
},
{
"time": "20:15:51",
"speaker": "Jonathan Jackson",
"text": "Alright. So the limita… this is about dealing with a limitation that comes from Mac OS 12 build, particularly."
},
{
"time": "20:15:59",
"speaker": "Dominik Kapusta",
"text": "Yes."
},
{
"time": "20:16:01",
"speaker": "Alex Martemyanov",
"text": "And bill times."
},
{
"time": "20:16:01",
"speaker": "Dominik Kapusta",
"text": "I'm Bill Times."
},
{
"time": "20:16:02",
"speaker": "Jonathan Jackson",
"text": "Got it. Okay. thanks for pointing it out. Uh, like I said, I think you might want to give people a little bit more time to respond to async if they have… particularly if they feel like the… Workspace thing is gonna be a hurdle, but my guess is it's not gonna be a big issue. Uh…"
},
{
"time": "20:16:22",
"speaker": "Alex Martemyanov",
"text": "Okay."
},
{
"time": "20:16:24",
"speaker": "Jonathan Jackson",
"text": "Next topic."
},
{
"time": "20:16:24",
"speaker": "Alex Martemyanov",
"text": "We can…"
},
{
"time": "20:16:26",
"speaker": "Jonathan Jackson",
"text": "From you, also."
},
{
"time": "20:16:26",
"speaker": "Alex Martemyanov",
"text": "Perfect. To the next topic then. Yeah, so, uh, this is another iteration of, uh, the Verfield project. I've been live coding for a while already. Uh, so this is the current state of things. It's changed a little bit, some of you may remember the first, uh, first demo. Uh, so the change is it's, uh, migrated to Go language, and, uh, now it runs on TART system that's used on, uh… We tried runners, I guess. And so the system is here to show you the available images, available runners and locations. So here you see I can have it. stored on the external SSD, because I have my internal 2TB ETH. up to completely, and I have the old MacBook Pro also managed from the same environment. And, uh, you can add other machines as well. Uh, so this is, uh, a wrapper around the TART system that manages, uh, Xcode installs, uh, SIP protection, and, uh, for virtual machine and real machines. Uh, if you use older hardware and, uh. It's, uh… basically providing MCP server for AI clients to. to run these UI tests. Mostly. on the older virtual machines. It can be not older virtual machines, but actual virtual machines. New refreshable machines as well, sir. Uh, for example, I don't update to macOS 27, because, uh… I expect some issues, but I use, uh, testing on macOS 27 using this, uh… a virtual machine system. And so basically the… Again, I have a question to the team in… In the light of a recent discussion of testing on order. macOS versions, for example, whether should I try to push it forward to, like, team-wide? So the idea… the idea is to be able, uh… again, we'll… And in the post-mortem. for the VS Code integration projects. So here you can see VS Code. Basically, it's running the Google browser. It can… run iOS browser as well, in simulator, and in addition, it can also run remote debugging, unlike Xcode, which had supported remote debugging. in a long time ago, but currently lacks the ability, so… Integrating these systems together allows, like, click one button. choose the virtual machine you want to run to test on, and it builds, deploys, and debugs in real time without any lags for library symbols transfer and other stuff. So it just works like you would do on your host machine, but in the virtual machine. So I wanted to hear opinions whether should be. try to adopt this team wide."
},
{
"time": "20:20:06",
"speaker": "Dominik Kapusta",
"text": "I can offer an opinion."
},
{
"time": "20:20:09",
"speaker": "Alex Martemyanov",
"text": "Nobody wants to."
},
{
"time": "20:20:09",
"speaker": "Dominik Kapusta",
"text": "uh… No, well, I mean… No one to be the only one commenting, but anyway. This will be the topic of, uh, of, uh, of the upcoming postmortem, uh, as well, that… The Apple engineering expectation of testing on older macOS version is essentially a dead low. We are supposed to test on older macOS versions, but we rarely do it. Perhaps for new features on macOS? I'm not sure. But certainly not for… Small… tasks. I don't think… well, I don't know, but I don't think we bother to test on all the ROS versions, every small version. Change to the UI, especially. And, uh… Because of that, if we had a way, and also we can't, we are unable to test in CI. So, if… I had a way to… Ask an agent to… run UI tests on a virtual machine. on an arbitrary OS version, that would be great. I would like to see the demo of that, actually. That would be awesome. And that is why we might be interested in adopting it. Perhaps."
},
{
"time": "20:21:41",
"speaker": "Alex Martemyanov",
"text": "Yeah, again, about the UI tests, correct point. So sometimes you want to run an agent reproducing an issue, for example, and it's not always convenient to upload it to CI, wait for the UI tests to be run. Look for the results, while if you run it locally on your machine, it will be blocked for 5 minutes, and do the stuff while you take a coffee. So this thing allows you to just run directly from the agent on the same machine in background and provide you screen recording with all this and the results."
},
{
"time": "20:22:22",
"speaker": "Jonathan Jackson",
"text": "Yeah, I mean, I think… You know, your point that our expectations are that we'd be testing on all versions that we support, but in practicality, I think almost nobody actually does that. And particularly with something like as old as Mac OS 12 where we don't have the hardware, we don't have like, most people don't have access to test it to your point, L. Um… I think I'd like to understand a little bit more about how the system would work, what kind of third-party dependencies there are, what it would look like to really set up and what it would cost. I think the idea of we would be able to test. On versions that we say we actually support is pretty appealing."
},
{
"time": "20:23:00",
"speaker": "Alex Martemyanov",
"text": "Okay, so probably the resolution here would be, um… So, first of all, I do use it as my self-made tooling, and uh… if… I can reach it to the point, I can make it to the point when you click a button, install the system, click a button, and it just runs. I think it would be worth. adopting at least as a selected auditory who can. have, uh, the means to testing, like, enough of RAM or some older hardware, uh, like, choose the early adopters and, uh, see how would we… expand it. And, uh, yeah, correct question by, uh, Federico about, uh, older ores. It's another topic that I wanted to raise during the post-mortem. During the incident, actually. Both for the incident, uh, that, uh, having… Maybe one dedicated machine, older machine, maybe somewhere that would run the UI test suite at least once per week for each release would be a great addition, even if it's like a physical machine somewhere. someone's office or home space."
},
{
"time": "20:24:28",
"speaker": "Jonathan Jackson",
"text": "Yeah, that's an interesting idea, too. I think… I guess I think that a proposal here that sort of lays out what a strategy for old testing would be that used this would be maybe a useful next step. Um, because then that would be a place that people could rally around, sort of solving that higher order problem of how do we actually make good on our promise of testing on the systems that we support. And it sounds like this would be one, but maybe not the only tool to consider, or maybe even as a combination. Does that make sense?"
},
{
"time": "20:25:00",
"speaker": "Alex Martemyanov",
"text": "Yep."
},
{
"time": "20:25:01",
"speaker": "Jonathan Jackson",
"text": "Cool. Okay, thank you. Interesting stuff. Sam, you're up next."
},
{
"time": "20:25:10",
"speaker": "SS",
"text": "Yeah, recently I have been chatting with some folks about being able to put Claude workflows in our CI. I talked about this with OT and they told us that they don't want Anthropic tokens being available in public repos. I've created a private repo to host the agents, uh, which is… Pretty bare bones right now. It lives here in the Apple Team Agents repo, it has CI set up, it has the Claude, uh, action set up. I have a workflow first for testing it, uh, just so we can make sure you can actually talk to Claude and back. Um, the only thing I'm waiting for is whether or not we should get a dedicated API key, because I've been told. Two different things from OT. So I'm just waiting to —. Get that sorted out. I think we should get a dedicated one. once we've done that, yeah, we can start doing things like an automation to clean up feature flags and such, like Fernando's, uh, proposed. So, uh, I'll post to Apple Diary once that's ready, but I think you'd be… if you do want to start building automations in this now, like, feel free, uh. As long as it's, you know, not super expensive. Pete, done."
},
{
"time": "20:26:19",
"speaker": "Pete Smith",
"text": "Yeah, thanks, Sam. This, this looks really cool. Just because I've seen, um, like, there's a few different conversations. There's a conversation with this, and there's the Bitrise, you know, running, um. uh… Agentic stuff on Bitrise as well. How does it all fit together? Like, I think I understand it, but, like, say I wanna… say we wanted to, you know, uh, build our feature overnight while we're in bed, um… How could we use this to do that? Or can we?"
},
{
"time": "20:26:43",
"speaker": "SS",
"text": "This is not that. This is definitely not that. This is not like the development environment thing. Yeah, these are not like the development machines. I really just expect these to be like GitHub Actions workflows that just do helpful things with Claude. Uh, and we just need a place to put those. Like, if it weren't for the API key limitation, this would live in the monorepo."
},
{
"time": "20:26:56",
"speaker": "Pete Smith",
"text": "Yep. Yeah, yeah, I guess, I guess what I'm wondering is, so, like, yeah, exactly, we can, we can hit Claude and, um, but, you know, Claude can't, well, build for us, so, like, well, build and check that it's actually working, so where would… say, say even, like, removing feature flag, where would that. On the PR, at the PR station."
},
{
"time": "20:27:17",
"speaker": "SS",
"text": "Yeah, this is something that Fernando and I was talking about, about doing some kind of coordination between these repos where the monorepo would talk back and forth, do the things that it needs to. This can create a branch. The monorepo can then do the actual testing. Yeah. Right now, I'm just using these for, like, dependability analysis kind of problems, or planning on anyway, which I have a demo for coming up soon."
},
{
"time": "20:27:33",
"speaker": "Pete Smith",
"text": "Okay. Yep. Cool. Yeah, cool. Thanks."
},
{
"time": "20:27:44",
"speaker": "SS",
"text": "Dominic."
},
{
"time": "20:27:46",
"speaker": "Dominik Kapusta",
"text": "Mm-hmm, uh… one of the limitations here is that GitHub Actions workflows are limited to 6 hours of runtime. So…"
},
{
"time": "20:27:53",
"speaker": "SS",
"text": "Also that, yeah."
},
{
"time": "20:27:54",
"speaker": "Dominik Kapusta",
"text": "Yeah. And also, it's, of course, non-interactive. Um… Which is why we're currently trying to figure out if Bitrise… or Bitrise or not, but anyway, looking for some infrastructure provider for, uh… macOS machines that could run Agentic workflows and be… and offer. interaction with them, right? So, uh… Bitrise RDE remote. development environments is one of the options there. Um… That is in progress. For now. They also… it's also quite a fresh feature on their end, and they have some bugs there that we're trying to resolve right now. And we have a CloudOps team member on that as well. the side of… Connecting them to ClickHouse or Sentry. Yeah, so… We'll see in some time."
},
{
"time": "20:28:58",
"speaker": "SS",
"text": "Yeah, yeah, to Dominic's point, like, that problem is much bigger than what this repo is. This really is just, I want to be able to run Claude on an automation, like I currently do on my own machine, and I'm just sick of doing it on my machine. I just want to push that up to GitHub Actions. Have it run on its own, and… Show up with PRs and Asana tasks occasionally, so. Okay."
},
{
"time": "20:29:19",
"speaker": "Jonathan Jackson",
"text": "So the limitations here are that it can… It can inspect code, it can run as an agent, but it can't build and run, so it would… prospectively submit a pull request, and then CIA would tell you whether it did the thing or not."
},
{
"time": "20:29:32",
"speaker": "SS",
"text": "Yeah, the idea that I'm thinking here is that, yeah, like PRs would go over back to the monorepo. This thing would open PRs against the monorepo. And then CI would do its thing over there. Yeah."
},
{
"time": "20:29:38",
"speaker": "Jonathan Jackson",
"text": "Right. Yeah, and if we needed to inspect it from there, we could pull a branch and do whatever else we needed."
},
{
"time": "20:29:46",
"speaker": "SS",
"text": "Yeah."
},
{
"time": "20:29:47",
"speaker": "Jonathan Jackson",
"text": "Okay, cool. Definitely sounds like a good step, even if you're not trying to solve everything all at once, which is reasonable. You're the next topic as well, Sam."
},
{
"time": "20:30:00",
"speaker": "SS",
"text": "Oh, is this the demo? Probably should have just done this all at once. Yeah. So one of the things that I."
},
{
"time": "20:30:04",
"speaker": "Jonathan Jackson",
"text": "Oh, before you… sorry, if you're doing demos, before we get into demos real quick… Uh, we had a note here that we were gonna actually record demos. Um, but we haven't ever done that. Mostly people have opted out of wanting to be recorded, so I wanted to just check in since we have 3 demos here. do we want to record these? Should we rely on the… just the transcript summaries at this point, and whatever tasks people have written? I'm kind of okay either way. Um… I don't know. I don't know that we've ever recorded a demo."
},
{
"time": "20:30:35",
"speaker": "Pete Smith",
"text": "Alessandro. Alessandro is waiting for his Wednesday morning. Won't be the same if we don't record our demo."
},
{
"time": "20:30:39",
"speaker": "Jonathan Jackson",
"text": "Right. That's the question, right? I know, yeah, it's like this is sort of a hard way to do this. So, Sam, would you like me to record or not?"
},
{
"time": "20:30:51",
"speaker": "SS",
"text": "I mean, I've already shown this to Alessandro, so I don't think he cares. Yeah. Up to you. I'm good either way. I remember. Don't worry. This is a super quick one anyway, just to show the initial problem that I'm trying to solve with… with a repo like this. Let me share my screen. Um, so I have, for a while, wanted the other half of the flaky tests project. Right now, we have a flaky test project, which keeps track of, uh, you know, tests that are failing. Um… I have traditionally been more interested in the problem of what are the workflows that are failing, uh, and why, because these things were just, like… this stuff comes up a lot, like, during Xcode upgrades in particular, so… This is something that is being used by Claude to pull the monorepo, get the actions, go and look at the code, figure out what's causing a problem, and report back and give a little write-up about what's going on. Um, the idea being that eventually you could then move something into this ready-to-fix section, and the agent would then go and actually open the PR later. Um, you know, this is just moving a… moving an automation that I've had on my machine is like a daily, like, 10 AM workflow for a while. and just putting it remotely so I don't have to think about it anymore. Which is the ultimate goal. So yeah, this is the kind of thing that I'm trying to do with this repo. Um… As an aside, OT is kind of working on… something along this as well, which I learned about as I was setting up this repo. They are setting up this template, which lives on Dub, for what it's worth, for these kind of analyzing report workflows. Umm. So that might be interesting if you are needing like access from things within the system already. Uh, like, I think Lucas is currently looking at getting that thing to have sentry access and all this kind of stuff, so… Yeah. Go check that project out if you're interested. That's it."
},
{
"time": "20:32:48",
"speaker": "Jonathan Jackson",
"text": "Awesome. Thank you. Pete, you're next."
},
{
"time": "20:32:57",
"speaker": "Pete Smith",
"text": "Yeah, thanks. Yeah, this is in the demo section. It's not really a demo, I guess, but just wanted to share. I'll speak a little bit about, um. ASTAC, so I put a link in… a link into the chat there, so I'm sure people have heard… of… PSTAC, or maybe people have heard of PSTAC. I'm going to just share my screen. So if you haven't, um… Pstack is a plugin created by an engineer at Cursor or SpaceX. It's basically just a set of skills that this engineer found were very good. at, um, fixing bugs, implementing features. So they created the PSTACK plugin. What ASTACK is, is basically just a port of PSTACK. So PSTACK was. built for cursor. So it has a lot of built in you know, instructions in terms of cursor models and how they work so forth. So I just. Ported PSTAC, uh, over… To, uh, ASTAC, um… Basically replacing any of the cursor references with Cloud Code and Codex references. So what AStack is, is a plugin that you can use in Codex or Cloud Code. It's like it sounds very abstract. It's not. It's really just a. A set of skills, a set of playbooks, some defined principles. And they all work together to what I found to just to produce good output. um… So I'll just, I'll talk a little bit in a moment about how ASTAG works. So how it's set up at the moment, you can see on the left, I just have it there under my public, under my own public repo. But if anybody wants to try it out, I would encourage it. You can see like I didn't do a lot of work here. I really want to undersell this. asked an agent to port this over. So take PStack, port it, make sure it works with Cloud Code and Codex. The most work I did was creating that logo you see on the left of the screen, so that's where all my creativity went into. Um… But yeah, you can go here, you can grab the plugin, you can install it in Cloud Code. And you can try it out. I'd recommend trying it out for… for bug fixes, even like small features that are well defined. So, you know, the evidence, like any evidence I have is just anecdotal. I've just found these particular set of skills, these set of playbooks and the principles that are defined just produce very good output. Again, anecdotal. I haven't measured this. I also think it seems to do it at lower token usage. But again, I haven't measured it. So how it works, how ASTAC or PSTAC work is, you know, you install the plugin, it sets up a hook, so at the start of your sessions, the agents are aware that ASTAC is there. And if you say, hey, build this feature or fix this bug or whatever, you can explicitly say. Enter potato mode. But that's actually part of the hook. The agent will know, oh, if you give it a feature or a bug, it will enter potato mode. All that is like a top level skill, which is basically a router. And depending on the task that you give it, like fix this bug or implement this feature. You will go and find the right playbook and follow that playbook and in the playbook. That would reference what skills to use, et cetera. So. again, you know, there's nothing, like, there's nothing magical here. I mean, there's been other plugins that have been very popular. I was talking to Fernando earlier, and we talked about… was it the Superpowers one? I think was another popular plugin. So things like this will probably be, as I was speaking to Fernando about, they'll probably be, can, you know, eventually end up in the harness. Probably won't need plugins like this at some point. But at the moment, this is something I'm using daily. And I find it very beneficial. It just results in really good output. So I would encourage people to try it out. Like, what what, you know, we've seen probably saw the post from Sebastian, from OM. He's creating, like, auto, which is their version of pstack. He just took pstack. And ported that and kind of made it very specific to Duck AI. That's like the the OM version of Pete's group. So if ASTAC, if people find it useful, we could basically turn that into like the Apple version of PSTAC, Apple team version of PSTAC. and… that's something we could do. So yeah, I would just encourage people to try it out, if you're interested. There's an Asana task there that I posted, and you can give any feedback, and if people want to. People find it useful and want to contribute and make it better. We can do that. It's under AI enablement AOR. I spoke for a long time, if anybody has. Sorry, I shared that artifact. Sorry, key. I didn't go through all this artifact."
},
{
"time": "20:37:59",
"speaker": "Pete Smith",
"text": "It's linked in… The Apple Weekly Task, so you can go and look at that artifact, it explains at a very high level how it works. Again, it's not magic, it's just a skill, a ruler, lots of other skills. Sorry, my dog is barking. I don't know if you can hear my dog, but. He's over there barking at me. That's very low. That's it. If anybody has any questions, you can ask me questions, or you can ask me questions later. Jane, sorry, your hand up."
},
{
"time": "20:38:35",
"speaker": "JF",
"text": "Yeah, sorry if you said already and I missed it. What changes have you made over the original piece, Doug?"
},
{
"time": "20:38:42",
"speaker": "Pete Smith",
"text": "Yeah, so… Oh, I'm sorry, it looks like. Okay, dog went dead. Um, yeah, that's why I wanted to undersell it. Like, I didn't, I didn't, I didn't do anything. So, uh, I created that logo. What, what, what I did, or what the agent did, was just port it over, support Pstack over, um, for Cloud Code and Codex. Like, the original Pstack has references to Cursor, and how Cursor. Works. So I just ported it over, so it works in… Um, codecs and cloud code. I didn't add anything additional yet. Um… But the idea will be we can add things if we find them useful. We can add Apple-specific things. If we find them useful. So yeah, no changes really. Sorry, one more thing, James. Thanks for reminding me. I'm going to reshare. Sorry, everybody. Very annoying. I'm going to reshare. So a good question. Like, PStack is being continuously updated. So I have a scheduled task that runs every Monday, looks at the latest PStack, and consumes the latest changes, and, like, ports over the latest changes. So I keep it up to date. So I look at the upstream and keep. a stack up to date. But that's other than that, I just I just port it. I don't make any other changes. Like, you know, of course, there's a chance it's not ported well. It might not work properly, but the intention is it's just a direct port."
},
{
"time": "20:40:09",
"speaker": "Jonathan Jackson",
"text": "Cool. Thank you, Pete. Paul, I'm protesting."
},
{
"time": "20:40:16",
"speaker": "Pau Ballart",
"text": "Yes, thank you. I'm going to share my screen. This is pretty much. a demo that I owe you of the snapshot testing. So, quick reminder, this is already available on iOS, not yet on macOS, because we are waiting for macOS 27. runners on big drives on the CI. on, uh… Yeah, pretty much. It's a package that has a very nice README explaining of it, what I will show today. So you can read it or tell your favorite agent to read it. But it has like requirements, how to use it. A bunch of examples for both iOS and macOS, even though, as I said, macOS. is disabled. But let's do a bit of an example. if I find the Xcode that just crashed, apparently. Ah, it's here, yeah. So, uh, how does it work? We start with the bill. Maybe, or maybe not, you will have previews. That is the new, like, hashtag… preview macro but for a snapshot so I created this one you don't need to use previews but it's just if you use them it becomes handy so there is this new preview provider that's the system. protocol, then if you conform it like this, so pretty much. You configure the different states that you want to have in the view. In this example, it's this segmented control. So there are two previews, one for each tab, let's say. You configure it here. So that's the model or the state of the view. And then you configure it here. So it will render one of the previews for each tab in this case. That basically gives you the previews here, but what is nice about it is that then you have the snapshot test. becomes basically a one liner or yeah one four liner uh but yeah just one call because it reuses the same previews that you refine it and then uh this is just a unit test like. Any other that we have. that runs the test. And this. Basically, you can configure it in different ways, for example. Here, if you want, um… to have a preview, but you don't want to snapshot it, because it's not needed. You have different. uh parameters here in this um. Let me show you. In these configurations, so there is… a scope where you can say, I want just for previews, just for snapshots, or for all of them. And then, um… in the test, you can snapshot in a different set of, um, configurations. So, in these asset snapshots. But everything is here in this package. We have different things. We have the different strategies. If you want to snapshot, for example, iPhones, iPads, all the combinations, just a single screen or a custom. Uh, size? And then you have also. the different sizes of components that you can have in an app, right? Majority of things are the screen, but you can also test some intrinsic content size, bottom sheets, or just any big size that you want. So all of that has, like, default parameters. In this example, for example, it's just, you know, normal screen, and I want to judge a snapshot in that mode, because that screen doesn't have light mode. So, it would work like this. And this is a unit test, it runs super fast, and uh… It passes in less than a second. how it works when you change something so let's say remember we have this previous let's say now i change this uh. this view. No, there is a design change, so we have this title here. If I run the tests, now they will fail. I will show you how it looks like in Xcode. So pretty much, issue recording, blah blah blah, does not match reference. And then, you can go here in your tests. We see the test that failed, and if you double-click, there is this gallery. That basically is showing us the reference, the failure, and the difference. So you can preview it like this. That's the original one. And then the new one that got recorded is this one with hello."
},
{
"time": "20:45:21",
"speaker": "Pau Ballart",
"text": "So you can see a diff that it's generated. Basically, you see everything is moving down. Um… So here it can happen two things. It's a real failure and you need to fix your UI, or it's a design change that is… expected, and you want to re-record the test flight, because this is now the new correct design. To do it, you have different ways. Here, there is a record Boolean. You can pass it to true. But at the same time, instead of changing code. If you go to the folder of the test. In in in Git you can let me show you. in a… in another screen. You can just delete the reference image in the file system. So that's the submodule that we have. In the, um… With all the snapshots. So if you go there and delete the file, it will recreate it on top of it. Um, so yeah, basically you have these two options. If there is no original file in the file system, it will just create it right away. If there is an existing one, it will do the comparison and so on. So the easiest usually is, um, to just remove. the image and re-record it. But let's say that… This works, and you upload a PR, and it fails in the CI. So how does it look like? Pretty much the current PR checks that we have for iOS, we'll report them here with the faillet, and it generates an HTML report, which is this one. that, uh… creates the same thing that we see in Xcode, but in the CI. So if you see a PR failing, you don't need to download the artifacts and so on. You just have to open this, and it has, like, these drop-downs. with the same concept, like the reference, the failure, and the diff. Um, so that's… A bit what I wanted to show. Then, um… So the snapshots live in a submodule, as you probably know. So it's a different repo. It's these DuckDuckGo Apple browser snapshots. And it just contains the PNG files. So that is this. Orchestration, if you want to run when you create snapshots, you need to open, let's say, two PRs. But there is a script that handles everything. So what you would do is here when you record your new snapshots. Maybe I can do it really quick. Just a second. I will. Show you exactly what I mean. So we have the submodule. Snapshot references. I'm gonna remove… The, um… Tests for this one. which is the scan QR code test. So I remove it from the file system. Okay, from this folder. And then when I record it or when I run the test, it will record. So first time it fails, but now it says no reference found on disk, which is fine. And then second time it runs, it already has the references and it passes. Okay, so now we want to open the PR, uh, and we can see that the snapshot reference module has the changes. Again, old and new with a new title. So there is a script in the scripts folder. around doo doo doo scripts. called Open Snapshot Zoom Module PR. You just run this over there, runs it, and it will open the PR, so commit, push, open PR, and so on. Um, and then once in your main branch or main PR, there is a new label that you can tag. Uh, let me show you here. Merge the snapshots. So this will merge the snapshots PR and update the pointer and so on. So, it's these two extra steps. You can do it manually, but, uh, the easiest is once you record it, and you have the changes here in Git, you just run the OpenPR, uh… script, and then once your main PR is approved, you can, uh, just use this label to merge the… the snapshots PR. So you…"
},
{
"time": "20:50:21",
"speaker": "Pau Ballart",
"text": "It's a bit of extra work, but I try to minimize it and automate as much as possible, so… That's how we currently have it set up with these failures and so on. Any questions so far?"
},
{
"time": "20:50:37",
"speaker": "Dominik Kapusta",
"text": "Yes, this is amazing, and do we have a documentation for that in the Apple documentation project?"
},
{
"time": "20:50:44",
"speaker": "Pau Ballart",
"text": "For the CAI part, you mean?"
},
{
"time": "20:50:46",
"speaker": "Dominik Kapusta",
"text": "Yeah, for…"
},
{
"time": "20:50:47",
"speaker": "Pau Ballart",
"text": "Or in general."
},
{
"time": "20:50:48",
"speaker": "Dominik Kapusta",
"text": "In general. You've mentioned that there's a script that simplifies the work, and that would be great to have it as a task in the Apple Documentations project, if it's not there yet."
},
{
"time": "20:51:00",
"speaker": "Pau Ballart",
"text": "Okay, I will add it, yeah. Yeah, so the biggest documentation right now is the README, but I understand that it's not, uh…"
},
{
"time": "20:51:02",
"speaker": "Dominik Kapusta",
"text": "Awesome. Thank you so much."
},
{
"time": "20:51:09",
"speaker": "Pau Ballart",
"text": "in the best place. I mean, it's useful, but…"
},
{
"time": "20:51:10",
"speaker": "Dominik Kapusta",
"text": "Mm-hmm."
},
{
"time": "20:51:12",
"speaker": "Pau Ballart",
"text": "Yeah, I will let it also in Asana."
},
{
"time": "20:51:16",
"speaker": "Dominik Kapusta",
"text": "Thanks so much."
},
{
"time": "20:51:16",
"speaker": "Jonathan Jackson",
"text": "Very cool, thank you, Pao."
},
{
"time": "20:51:20",
"speaker": "Pau Ballart",
"text": "I have one more thing to share, which is… So two things. Why is this useful, right? Like, because, okay, we can add a snapshot test and so on. There are three things that I see that are important. First one is PR reviews. Since now the two PRs are linked, it makes it easier to review changes, right? You read UI code of SwiftUI, for example, and you see paddings, frames, blah, blah, blah. Seeing a screenshot helps a lot, so now it would be easy to compare side by side, because you will have here, um… For example, this one. The… the file changes. Directly in the… well, it's not the… This one needs a rename. And you will have the file changes on GitHub to review side by side like this. So PR reviews are better than. I think it was already mentioned we can. think about our UI tests, and if we have UI tests that are asserting. Just how things look like. It's a good candidate to replace with. with the snapshots. Um, so… I will open a proposal at some point, but the idea is… Since UI tests are flaky and snapshots and units. are less flaky and run quite fast. We might look at opportunities to move some of these tests and with a good combination of unit and snapshots. Yes, we are still missing the navigation part of I tap this button and this other screen opens. But yeah, it's up for discussion, but I think you have already a good coverage of a feature, right? And the last thing is ship reviews, because here in this other repo, it's becoming basically like an S3 bucket of data. PNG files. So when you prepare your ship review, if you have used this, you don't need, uh… to regenerate the snapshots and so on. You would just even link it from here or pick them from here. And that's just a starting point. We can improve it with future ideas, like we were talking about different languages. So how we could snapshot test different languages, different dynamic sizes for accessibility, like a large text size and so on. Even, I don't know, when the snapshots PR is open. with some automation, check the original Figma design, and compare it, and there are, there are, um… ways to improve this. So this is just a starting point. And last thing I want to mention, it was part of the tech design to track adoption of this. So far, I'm the only one contributing, which is why I'm doing this demo as well. But I created with Claude an automation that every week checks the repo. It's very easy, because we can just go to the insides of this, um… of this repo and see contributors, but also just check, uh, the asset and snapshot, uh, usages. So this is a dashboard that I shared, uh, within DuckDuckGo. It's updated every week, and basically we can see how. how it grows over time. Um… So yeah, if you fancy, please use it. And if there are any issues, let me know. I'm like the unofficial or official DRI of this. Happy to be on top of it if it's causing issues as well."
},
{
"time": "20:54:59",
"speaker": "Jonathan Jackson",
"text": "It's really cool, Pat. Thank you for sharing. Yes."
},
{
"time": "20:54:59",
"speaker": "DR",
"text": "Good work, pal."
},
{
"time": "20:55:04",
"speaker": "Jonathan Jackson",
"text": "Very cool. We are now at 5 minutes to the top of the hour. Dominic and Alex, you both have postmortems on the list. Do you want to… run those now, understanding we'll go past time, and anybody who can stay is welcome to stay or not? Or would you rather find another time for those?"
},
{
"time": "20:55:24",
"speaker": "Dominik Kapusta",
"text": "I'm fine running it. It should also be short, but not sure about Alexa's one."
},
{
"time": "20:55:33",
"speaker": "Alex Martemyanov",
"text": "Probably can wait till the next Apple Weekly."
},
{
"time": "20:55:37",
"speaker": "Jonathan Jackson",
"text": "Okay."
},
{
"time": "20:55:37",
"speaker": "Alex Martemyanov",
"text": "I think it was waiting for a year and it didn't wait another week."
},
{
"time": "20:55:41",
"speaker": "Jonathan Jackson",
"text": "All right."
},
{
"time": "20:55:42",
"speaker": "Alex Martemyanov",
"text": "Thank you."
},
{
"time": "20:55:43",
"speaker": "Jonathan Jackson",
"text": "Thank you, Alex. Um, then, Dominic, why don't you go ahead and get started? Anybody who's not interested in Dominic's incident post-mortem, obviously, is welcome to drop. If we run a little bit late and you need to be somewhere else, that's also okay to drop. And Alex, I'll keep yours on the agenda for next week. Thank you."
},
{
"time": "20:56:01",
"speaker": "Dominik Kapusta",
"text": "Thanks. I've dropped the link in the agenda as well for the incident postmortem. What happened last Monday is we released the app that was crashing on macOS 12. And, uh, the app was out… well, uh, so the app was out for, like. 3 hours before we, uh, realized that there is a crashing scenario on macOS 12, and then that was handled real quick, uh, with app store rollout stopped. And, uh, Sparkle rollout stopped, uh, as well at some point, um, and the fix was released, actually… well, it… It. Well, almost immediately on DMG, and the next day on App Store, because of the review time. Um, but, uh, how it happened? So, it happened, uh, because we had a change to, um, the save credentials pop-up, which shows up every time you're offered to… you're being offered to save a password. Um… On macOS 12, um… And that change was due to dropping a storyboard, replacing a storyboard with a code-initialized view. So what didn't go as planned? This, uh, change was not tested on old OS versions before merging, and it was strictly macOS 12 issue. Uh, like I mentioned before in this call, we had that engineering expectation in place, but we're not really enforcing it for every change. There's no way of enforcing it, uh, and there's no way of. testing, um… all those versions in CI, so… that's why it happened. Really. Uh, the other thing is, uh, we didn't know about the crash, because, uh, I overlooked it in Sentry. I was the release DRA and I overlooked it in Sentry. There was a single crash on macOS 12 there. Uh, that I treated as noise, and I went ahead and released the app. The sentry summaries weren't generated on that day, because we had another… Whoopsie. Um, the week before, when we updated, uh. Well, some updated the model version and I reviewed it and approved it. and we merged that change, and we updated the model version, but we didn't update the SDK version, and the old SDK didn't… wasn't recognizing the new model, and it failed to… collaborate, so we didn't generate summaries. Um… So that's the other… the other thing. Uh, when the… but! When the summary was generated, uh, it clearly pointed out that there is a crashing scenario on Mac OS 12, which started the incident. A week of internal testing didn't catch a relatively easy-to-reproduce crash. It was a 100% crashing scenario, which wasn't caught. Because we don't have testers on macOS 12. Well, apparently we have one, because we got this one crash at some point. I don't know who it is. I'm…"
},
{
"time": "20:59:33",
"speaker": "Alex Martemyanov",
"text": "Yeah, and by the way, there was another kind of related issue about the crash on Intel machines. It's not directly related, but yeah, it was close."
},
{
"time": "20:59:41",
"speaker": "Dominik Kapusta",
"text": "Yes, another one, exactly. We also don't have testers, or many testers, on x86. architecture. The App Store review took 15 hours, which is rather long for an expedited review, but also there's nothing we can do about it. At least it got in the queue… I mean, it got in the review pretty quickly after requesting a completed review. What went well is that the sentry summary that was eventually generated, um… Clearly identified the crash, and suggested addressing it, and the fix was simple, quick to prepare and test, it was just assigning an initial frame to a view. And the hotfix release automation workflow less at this time. It's always important to point this out, because… We only talk about CI when it's not working correctly. Despite the long review time, we only had 37 crashes in total and only 9 affected users. That's all. for action items and DRIs, we actually, uh… there was an action item to update agent's instruction prefect… essentially update prefect to ensure that Claude agent SDK version is… compatible with used models. Uh, Sam has handled it, and, uh, well, there is a code command right now to, to update the SDK version whenever you update the model version. Um… The other topic that's not actually mentioned here is what to do about testing on older OS versions, but this is something we will be handling separately through Alex's work, mentioned earlier on the call. And… That's it. Finally, I created an incident task through the form. Asana forum, and marked the incident as high, because it's a crashing scenario, we need a hotfix, and an SRE person joined the call, and we talked a little bit, and he, um… Explain to me that if it's a one-person rollout. It's not a high severity incident, and we dropped it to medium, and they dropped the call. And that's it. And, um… That's something to keep in mind for later, for next incidents, when they are only starting rolling out, we may not need SRA help there. This is all I have."
},
{
"time": "21:02:09",
"speaker": "SS",
"text": "Regarding the action item, are you happy enough with just having the code comment now that reminds about the Claude SDK?"
},
{
"time": "21:02:15",
"speaker": "Dominik Kapusta",
"text": "I think so."
},
{
"time": "21:02:16",
"speaker": "SS",
"text": "you know, it's… it's not bulletproof, like, what I… what I wanted to do was to just ask the Claude SDK, what models do you support, and check that it's in the list, but it only includes… yeah, it doesn't include all the actual supported models."
},
{
"time": "21:02:18",
"speaker": "Dominik Kapusta",
"text": "I think, yeah. I saw that, yeah."
},
{
"time": "21:02:28",
"speaker": "SS",
"text": "Eh, maybe the comment's okay for now. I tried to make it very obvious what to do and how to confirm this."
},
{
"time": "21:02:32",
"speaker": "Dominik Kapusta",
"text": "Yeah, I mean, that's okay. Also, that comment would show up, because it's just very close to the line that is changing, it will show up in the GitHub UI when reviewing the code, so it should be fine."
},
{
"time": "21:02:37",
"speaker": "SS",
"text": "It's right there. Yeah, the only way I was considering extending this was to just have a GitHub Actions check that looks for model changes and adds a comment, GitHub Actions comments. But I would want to do this for all of Prefect. Whatever. I'll look at that."
},
{
"time": "21:02:56",
"speaker": "Dominik Kapusta",
"text": "Yeah, yeah. Should be fine. Okay."
},
{
"time": "21:02:58",
"speaker": "SS",
"text": "Okay."
},
{
"time": "21:03:00",
"speaker": "Dominik Kapusta",
"text": "Thank you. Thanks for handling that, Sam. Alex?"
},
{
"time": "21:03:05",
"speaker": "Alex Martemyanov",
"text": "Yeah, I got a proposal in… I got it related to post-mortem of the VS Code, but I think it more suits here. What about adding… updating our process to require adding UI tests in addition to renting some older Mac to run older OSs, but make a requirement to the process to cover any UI change. with a UI test. Even if it's a fix or a quick addition in the uncovered region, especially having all the Agentic workflows, it may be a quick and cheap option. I mean, cheap to add. maybe not to maintain. But it adds, of course, it adds extra cost for the runners. Or maybe we could have like a test suite number two that runs weekly, for example, and not daily."
},
{
"time": "21:04:11",
"speaker": "Dominik Kapusta",
"text": "I actually think it's… Some Windows team uses something similar. whenever they change UI, they add UI tests, and they have plenty of UI tests, and that's another problem. I believe, but… Yeah, we could… We could discuss it, I guess."
},
{
"time": "21:04:31",
"speaker": "Alex Martemyanov",
"text": "Yeah, because, uh…"
},
{
"time": "21:04:31",
"speaker": "Jonathan Jackson",
"text": "I think that's what we're considering. Sorry, go ahead."
},
{
"time": "21:04:35",
"speaker": "Alex Martemyanov",
"text": "Because obvious, like, obvious… thing that could fix it is, uh, like, QA, which we are missing, for… everybody knows for… for why, for what reasons, uh, but we have a lot of, uh, supported platforms, a lot of supported macOS versions, and, uh, it's, uh… Like, uh, it's impossible to… make the requirement as in the mentioned. Uh, process, like, to test all the supported macOS versions, because not everybody even have all these machines that, like, I have the older one, but it's not DuckDuckGo, it's my personal machine. And, uh, I can test it, but it takes time. Uh, and nobody… nobody does it, frankly. And we have no QA, and it's impossible to guarantee the quality in such conditions. So, maybe it's required. It should be a required condition to… to… raise the UI test coverage, the end-to-end coverage, so we can validate we are actually producing quality UI tests. Stuff."
},
{
"time": "21:05:50",
"speaker": "Jonathan Jackson",
"text": "Yeah, I think that's worth considering alongside the virtual machine testing stuff that you were talking about earlier, right? I mean, we've known for a long time that we clearly have testing gaps, especially on older versions and older hardware. And… To this point, I think we've just accepted that as the way it is and every now and then something like this goes wrong and we deal with it. um… But we should definitely think about whether there's a promise we're making to our users there that we're not able to fulfill properly. And I think certainly that's true right now. The question is, what does it take to do better at that promise? So I'd love to see some proposals around moving some of these pieces forward."
},
{
"time": "21:06:32",
"speaker": "Dominik Kapusta",
"text": "All right, that was all for me. Thank you very much."
},
{
"time": "21:06:38",
"speaker": "Jonathan Jackson",
"text": "Alright, and thank you. Um, so, Alex, we're gonna wait till next week for your other postmortem. Thank you for your flexibility there. And thank you all who stayed to finish that out. I appreciate it. And I look forward to seeing you again soon. Bye, everyone."
},
{
"time": "21:06:54",
"speaker": "Alex Martemyanov",
"text": "Thank you. Bye."
}
]
}
@@ -14,3 +14,137 @@ source: "https://us01mynotes.zoom.us/gBBa-iiUQLycw55WRV1tnw"
- **20:03:14 · Jonathan Jackson:** Hello, everyone. Let me go ahead and get started. Quick runs through our scheduled rotations as usual. This week, iOS triage is L, followed by Bartek, and then Brindy. Mac OS triage on is on this week by Jorge and then Rachel. Uh, for iOS release, Sam, you're on duty this week, followed by Bartosh, and then Mihao. And for macOS release, Dominic was last week, hasn't closed that task yet. Tom, you're on this week, followed by Juan, and then Joseph. You all may have also seen, I published our… Uh, release schedule changes around the winter holidays, uh, last week. Um, next step there is for me to look at out-of-office stuff and start trying to coordinate and… uh, coordinate swaps and coverage during that time when a lot of us are gonna be out, so keep an eye out for that soon, I'll get that. Um… I'm also using the Zoom transcript feature on top of Zoom summaries, and I'm gonna feed the transcript to Claude and see if we get a better summary that way than we have been out of Zoom's summarization AI. Uh, so… And I'll share maybe both, or I'll see what it comes up with, and I'll share an update after this meeting about what we ended up with there. Um, and one final reminder, next week, uh, is performance review week. The process starts with self-reviews happening on Monday. And then there's a few days of CAs writing reviews, and then a few more days of calibration after that. um… But because there's really only one day in that official schedule dedicated to self-eval writing, I suggest y'all think about starting this week. If you're anything like me, I usually need to write it, and then think about it, and then come back and edit it heavily, and then think about it, and then write it again. So I would suggest giving yourself the extra time if you need to do that. to get started early. Tom, what's up?
- **20:05:23 · Tom Strba:** Just a small note, I checked the macOS release rotations. Yes, there is not finished one rotation that's Dominic's, but this week is Diego. Right? Is that true?
- **20:05:36 · Jonathan Jackson:** Mm, got it.
- **20:05:37 · Tom Strba:** And I think I'm next week. Because I almost got a heart attack when you said that I am…
- **20:05:40 · Dominik Kapusta:** Yes, this week is Diego. And sorry for not closing my task.
- **20:05:41 · Jonathan Jackson:** Okay.
- **20:05:45 · Tom Strba:** that I am this week.
- **20:05:45 · Jonathan Jackson:** This week… Why is Diego… I don't see a task… this is a macOS release we're talking about, right?
- **20:05:52 · Tom Strba:** Yeah, I think Diego maybe marked it as complete in the beginning of the week, because there's just, like, triage…
- **20:05:57 · Jonathan Jackson:** Yes, you're right, Tom, your block is the 26th to the 30th, because that task is missing, I didn't see it, I wasn't looking at the dates, I was just reading the list.
- **20:06:04 · Tom Strba:** That's okay. Just wanted to be sure.
- **20:06:07 · Jonathan Jackson:** Okay. Diego is responsible for the release this week. It sounds like he's actually doing that. Is that right?
- **20:06:08 · Tom Strba:** Perfect.
- **20:06:14 · Jonathan Jackson:** He knows. Okay, cool. Thank you.
- **20:06:14 · Dominik Kapusta:** That's right.
- **20:06:18 · Jonathan Jackson:** Um, no problem. Okay. Anyway, think about starting your self reviews this week. If you feel like you're gonna need more than a day to knock it out. Bartek, over to you.
- **20:06:31 · Bartek Waresiak:** Yep, I will be stepping down as IS app DRI. Jacek will take the primary role, and Brindy will take on secondary role. So, yeah, if there is IS drama, if there are IS issues, if there are any urgent things. Go to them! And thank you very much.
- **20:06:57 · Jonathan Jackson:** All right. Thank you. An, you're next on the agenda.
- **20:07:08 · Anh Do:** Oh, that was quick. Yeah, so in OK, there have been a few projects that involve native changes required as part of the front-end-led projects. Um, so the background is that they, uh, want to contribute some PR, and, uh, um… AI assisted Pi, of course, and then what have been working for us lately is like depends on the size of the task like I would be the one to like either. Dri the change that they are they want to make, or like they prepare the Pi, and I do a thorough review of that. So and now we want to kind of do this more regularly. And so there was a need for some kind of like guidelines that we can give to contributor from outside the apple team. So I was just starting like a small task to basically collect ideas from people. And if you have anything that we already have somewhere, maybe I could get put in. And so. it becomes something that we can point other people to when they want to contribute to the Apple codebase to ensure our standard and make sure that, like, everything is being done the way that we want. So that's about it. Nice.
- **20:08:19 · Jonathan Jackson:** Cool, thank you. I just took a quick peek. I see you do mention the engineering expectations. I think that's a good source. We might also want to link… to the onboarding process. Not that they would need to follow all of it, but some of… that's basically our best source of, like, how to get set up with the codebase. So there's probably a little work to do to figure out how to have a slimmed down version of that that folks could follow. And I'm sure other people have other ideas as well. So thank you. Um… Luke, we have a non-Apple person lurking in our midst. Welcome. What can we do for you?
- **20:08:54 · Luke Andrew:** Hello. Yes, thanks for having me. Luke from SRE, for people that I haven't met before. Um, this is just a plea for generic Apple help. I don't want to, sort of, get all of your attention to focus on this task. We basically have… a looping client that seems to be sending, uh, requests once a second to us, uh, that is Apple, but not DuckDuckGo Apple, based on the user agent, and doesn't seem to be Safari either. Um… And it's cumulatively across lots of different devices, lots of different queries, sending in the region of 3 million searches a day. Which is substantial, not sort of catastrophic, but substantial enough searches that it's something that we want to deal with. Umm. And I don't really know where to go next to try and understand where it might be coming from or how we might try and reproduce it. So I don't want to solve that in this call, but I just wanted to bring your attention to the task. And if there's anyone who thinks they might be able to help, I'd really appreciate. a second pair of eyes, and pointed out that it's not even worth, with what we have at the moment, that we can, like, escalate this to Apple, because we just don't have enough, uh, information to get any help from them. So yeah, that's it. If anyone has any questions immediately or async, just pop them on the task. Yeah, that's all I have.
- **20:10:18 · BA:** Quick one, Luke.
- **20:10:18 · Jonathan Jackson:** This is not from a single client, but from a particular app version, potentially, is that right?
- **20:10:24 · Luke Andrew:** Yeah. Yeah.
- **20:10:24 · Jonathan Jackson:** Okay. You know, be scared that it was one client going crazy. Okay, um…
- **20:10:30 · BA:** Yeah, I was going to ask, what's the — we eliminated bots from the equation. Are we happy that it's not bots?
- **20:10:36 · Luke Andrew:** It doesn't look like bots. The queries all look human. The rate is really low compared with bots. The moment that I put in a block for it and it immediately stopped, like redirected away.
- **20:10:39 · BA:** Okay. Right.
- **20:10:52 · Luke Andrew:** It got, like, a 302 from deep, and then just immediately stopped, so that doesn't feel like… bossy behavior either. They normally persist a little bit and then give up. Whereas this was just immediately, it broke out of whatever loop it was in and stopped.
- **20:11:09 · Jonathan Jackson:** Interesting.
- **20:11:10 · Luke Andrew:** Uh, so don't all take a look, because that would be a waste of time, but if you think you can help, um, I'd appreciate a pair of eyes. I'm gonna drop off. Thank you all for having me.
- **20:11:19 · Jonathan Jackson:** Yeah, one last quick question. It sounds like you're treating this as something that definitely needs attention, but not an incident at this point. Is that correct?
- **20:11:25 · Luke Andrew:** Yeah, that's correct. It's been going on for… Based on… um, how we categorize queries for, um, like, ads perspectives. These get classified as a separate category based on user agents, so we don't send them to Bing and say, these are definitely good queries. Um, it's been happening for about a month. So it's, at the moment, not urgent. I'm gonna raise it with SRE, maybe, and see if we think it should be an incident that we just let run, but at the moment, that's not how we'.
- **20:11:55 · Jonathan Jackson:** Okay. Great. Thank you for the clarity there. Uh, and yeah, if anybody thinks they can help Luke out and figure out what's going on there, feel free to hop into the task. Thanks for coming. Moving on. Alex, you're first in the regular topics list.
- **20:12:16 · Alex Martemyanov:** Yeah, hey. I have several topics for today, including the VS Code integration postmortem to run later. And the first one goes with a proposal. The proposal is pretty short. And kind of related to the, uh, recent incident with Mac OS 12 crash. that we are not. Testing for macOS 12 and 13, I guess, because we are missing the infrastructure on CI runners. And the suggestion is pretty simple to extract the macOS UI tests target from the main macOS Xcode project, leaving it as a separate macOS project inside the. uh, Xcode workspace. So it will be available the same way it is now, and, uh, runnable. So only, uh, the only issue I can see for those who use, uh, the actual macOS Xcode project instead of the workspace. they would have to switch to another project to run the tests. So, I just wanted to hear if there is any concerns or objectives against it.
- **20:13:39 · Dominik Kapusta:** I think some time ago, we… someone asked… who uses or doesn't use Workspace, and people who didn't use Workspace were in a… In a minority, anyway, like, very few of them, so… I guess… I could move to the workspace.
- **20:14:06 · Jonathan Jackson:** I think it'd be good to ask that question in that proposal, at least more explicitly, and just give people a chance to chime in if they feel like that would be a hurdle. Um… Other question for me is, is there… should we think about the iOS side of this as well, and to match… structure, or is this really a macOS only question?
- **20:14:26 · Alex Martemyanov:** Okay. Maybe, I… Didn't think about it, actually. And the main concerns here is the compatibility. So we can reach macOS 12 compatibility, at least for some local runners that I'm going to present further. And the build times, so we have package fetching and building for every macOS runner that runs the UI tests and any of the dependencies. None of the dependencies are used during the runs, so it's a 10-second step for each runner that can be easily omitted, and so, yeah. And yeah, it's not related to the main browser targets, so there's no risks. Uh, and they're both, uh, U.S. Probably to think about it in the ER.
- **20:15:25 · Dominik Kapusta:** I think iOS is not…
- **20:15:25 · Jonathan Jackson:** I mean, I don't have a strong… sorry, I don't have a strong feeling there, I just felt like if we're gonna make this change, like, if it's gonna end up with different structures depending on the app, we should think about whether we really want that to be the case, or if we should do the other.
- **20:15:37 · Alex Martemyanov:** No, it actually shouldn't change any structure, any file structure, just the project placement.
- **20:15:38 · Jonathan Jackson:** Screwed up. Yeah, yeah, yeah. Dominic, I interrupted you. I'm sorry.
- **20:15:44 · Dominik Kapusta:** Ah, no worries, I just wanted to say that iOS is not affected by this at all, because… iOS doesn't need testing on various macOS versions.
- **20:15:51 · Jonathan Jackson:** Alright. So the limita… this is about dealing with a limitation that comes from Mac OS 12 build, particularly.
- **20:15:59 · Dominik Kapusta:** Yes.
- **20:16:01 · Alex Martemyanov:** And bill times.
- **20:16:01 · Dominik Kapusta:** I'm Bill Times.
- **20:16:02 · Jonathan Jackson:** Got it. Okay. thanks for pointing it out. Uh, like I said, I think you might want to give people a little bit more time to respond to async if they have… particularly if they feel like the… Workspace thing is gonna be a hurdle, but my guess is it's not gonna be a big issue. Uh…
- **20:16:22 · Alex Martemyanov:** Okay.
- **20:16:24 · Jonathan Jackson:** Next topic.
- **20:16:24 · Alex Martemyanov:** We can…
- **20:16:26 · Jonathan Jackson:** From you, also.
- **20:16:26 · Alex Martemyanov:** Perfect. To the next topic then. Yeah, so, uh, this is another iteration of, uh, the Verfield project. I've been live coding for a while already. Uh, so this is the current state of things. It's changed a little bit, some of you may remember the first, uh, first demo. Uh, so the change is it's, uh, migrated to Go language, and, uh, now it runs on TART system that's used on, uh… We tried runners, I guess. And so the system is here to show you the available images, available runners and locations. So here you see I can have it. stored on the external SSD, because I have my internal 2TB ETH. up to completely, and I have the old MacBook Pro also managed from the same environment. And, uh, you can add other machines as well. Uh, so this is, uh, a wrapper around the TART system that manages, uh, Xcode installs, uh, SIP protection, and, uh, for virtual machine and real machines. Uh, if you use older hardware and, uh. It's, uh… basically providing MCP server for AI clients to. to run these UI tests. Mostly. on the older virtual machines. It can be not older virtual machines, but actual virtual machines. New refreshable machines as well, sir. Uh, for example, I don't update to macOS 27, because, uh… I expect some issues, but I use, uh, testing on macOS 27 using this, uh… a virtual machine system. And so basically the… Again, I have a question to the team in… In the light of a recent discussion of testing on order. macOS versions, for example, whether should I try to push it forward to, like, team-wide? So the idea… the idea is to be able, uh… again, we'll… And in the post-mortem. for the VS Code integration projects. So here you can see VS Code. Basically, it's running the Google browser. It can… run iOS browser as well, in simulator, and in addition, it can also run remote debugging, unlike Xcode, which had supported remote debugging. in a long time ago, but currently lacks the ability, so… Integrating these systems together allows, like, click one button. choose the virtual machine you want to run to test on, and it builds, deploys, and debugs in real time without any lags for library symbols transfer and other stuff. So it just works like you would do on your host machine, but in the virtual machine. So I wanted to hear opinions whether should be. try to adopt this team wide.
- **20:20:06 · Dominik Kapusta:** I can offer an opinion.
- **20:20:09 · Alex Martemyanov:** Nobody wants to.
- **20:20:09 · Dominik Kapusta:** uh… No, well, I mean… No one to be the only one commenting, but anyway. This will be the topic of, uh, of, uh, of the upcoming postmortem, uh, as well, that… The Apple engineering expectation of testing on older macOS version is essentially a dead low. We are supposed to test on older macOS versions, but we rarely do it. Perhaps for new features on macOS? I'm not sure. But certainly not for… Small… tasks. I don't think… well, I don't know, but I don't think we bother to test on all the ROS versions, every small version. Change to the UI, especially. And, uh… Because of that, if we had a way, and also we can't, we are unable to test in CI. So, if… I had a way to… Ask an agent to… run UI tests on a virtual machine. on an arbitrary OS version, that would be great. I would like to see the demo of that, actually. That would be awesome. And that is why we might be interested in adopting it. Perhaps.
- **20:21:41 · Alex Martemyanov:** Yeah, again, about the UI tests, correct point. So sometimes you want to run an agent reproducing an issue, for example, and it's not always convenient to upload it to CI, wait for the UI tests to be run. Look for the results, while if you run it locally on your machine, it will be blocked for 5 minutes, and do the stuff while you take a coffee. So this thing allows you to just run directly from the agent on the same machine in background and provide you screen recording with all this and the results.
- **20:22:22 · Jonathan Jackson:** Yeah, I mean, I think… You know, your point that our expectations are that we'd be testing on all versions that we support, but in practicality, I think almost nobody actually does that. And particularly with something like as old as Mac OS 12 where we don't have the hardware, we don't have like, most people don't have access to test it to your point, L. Um… I think I'd like to understand a little bit more about how the system would work, what kind of third-party dependencies there are, what it would look like to really set up and what it would cost. I think the idea of we would be able to test. On versions that we say we actually support is pretty appealing.
- **20:23:00 · Alex Martemyanov:** Okay, so probably the resolution here would be, um… So, first of all, I do use it as my self-made tooling, and uh… if… I can reach it to the point, I can make it to the point when you click a button, install the system, click a button, and it just runs. I think it would be worth. adopting at least as a selected auditory who can. have, uh, the means to testing, like, enough of RAM or some older hardware, uh, like, choose the early adopters and, uh, see how would we… expand it. And, uh, yeah, correct question by, uh, Federico about, uh, older ores. It's another topic that I wanted to raise during the post-mortem. During the incident, actually. Both for the incident, uh, that, uh, having… Maybe one dedicated machine, older machine, maybe somewhere that would run the UI test suite at least once per week for each release would be a great addition, even if it's like a physical machine somewhere. someone's office or home space.
- **20:24:28 · Jonathan Jackson:** Yeah, that's an interesting idea, too. I think… I guess I think that a proposal here that sort of lays out what a strategy for old testing would be that used this would be maybe a useful next step. Um, because then that would be a place that people could rally around, sort of solving that higher order problem of how do we actually make good on our promise of testing on the systems that we support. And it sounds like this would be one, but maybe not the only tool to consider, or maybe even as a combination. Does that make sense?
- **20:25:00 · Alex Martemyanov:** Yep.
- **20:25:01 · Jonathan Jackson:** Cool. Okay, thank you. Interesting stuff. Sam, you're up next.
- **20:25:10 · SS:** Yeah, recently I have been chatting with some folks about being able to put Claude workflows in our CI. I talked about this with OT and they told us that they don't want Anthropic tokens being available in public repos. I've created a private repo to host the agents, uh, which is… Pretty bare bones right now. It lives here in the Apple Team Agents repo, it has CI set up, it has the Claude, uh, action set up. I have a workflow first for testing it, uh, just so we can make sure you can actually talk to Claude and back. Um, the only thing I'm waiting for is whether or not we should get a dedicated API key, because I've been told. Two different things from OT. So I'm just waiting to —. Get that sorted out. I think we should get a dedicated one. once we've done that, yeah, we can start doing things like an automation to clean up feature flags and such, like Fernando's, uh, proposed. So, uh, I'll post to Apple Diary once that's ready, but I think you'd be… if you do want to start building automations in this now, like, feel free, uh. As long as it's, you know, not super expensive. Pete, done.
- **20:26:19 · Pete Smith:** Yeah, thanks, Sam. This, this looks really cool. Just because I've seen, um, like, there's a few different conversations. There's a conversation with this, and there's the Bitrise, you know, running, um. uh… Agentic stuff on Bitrise as well. How does it all fit together? Like, I think I understand it, but, like, say I wanna… say we wanted to, you know, uh, build our feature overnight while we're in bed, um… How could we use this to do that? Or can we?
- **20:26:43 · SS:** This is not that. This is definitely not that. This is not like the development environment thing. Yeah, these are not like the development machines. I really just expect these to be like GitHub Actions workflows that just do helpful things with Claude. Uh, and we just need a place to put those. Like, if it weren't for the API key limitation, this would live in the monorepo.
- **20:26:56 · Pete Smith:** Yep. Yeah, yeah, I guess, I guess what I'm wondering is, so, like, yeah, exactly, we can, we can hit Claude and, um, but, you know, Claude can't, well, build for us, so, like, well, build and check that it's actually working, so where would… say, say even, like, removing feature flag, where would that. On the PR, at the PR station.
- **20:27:17 · SS:** Yeah, this is something that Fernando and I was talking about, about doing some kind of coordination between these repos where the monorepo would talk back and forth, do the things that it needs to. This can create a branch. The monorepo can then do the actual testing. Yeah. Right now, I'm just using these for, like, dependability analysis kind of problems, or planning on anyway, which I have a demo for coming up soon.
- **20:27:33 · Pete Smith:** Okay. Yep. Cool. Yeah, cool. Thanks.
- **20:27:44 · SS:** Dominic.
- **20:27:46 · Dominik Kapusta:** Mm-hmm, uh… one of the limitations here is that GitHub Actions workflows are limited to 6 hours of runtime. So…
- **20:27:53 · SS:** Also that, yeah.
- **20:27:54 · Dominik Kapusta:** Yeah. And also, it's, of course, non-interactive. Um… Which is why we're currently trying to figure out if Bitrise… or Bitrise or not, but anyway, looking for some infrastructure provider for, uh… macOS machines that could run Agentic workflows and be… and offer. interaction with them, right? So, uh… Bitrise RDE remote. development environments is one of the options there. Um… That is in progress. For now. They also… it's also quite a fresh feature on their end, and they have some bugs there that we're trying to resolve right now. And we have a CloudOps team member on that as well. the side of… Connecting them to ClickHouse or Sentry. Yeah, so… We'll see in some time.
- **20:28:58 · SS:** Yeah, yeah, to Dominic's point, like, that problem is much bigger than what this repo is. This really is just, I want to be able to run Claude on an automation, like I currently do on my own machine, and I'm just sick of doing it on my machine. I just want to push that up to GitHub Actions. Have it run on its own, and… Show up with PRs and Asana tasks occasionally, so. Okay.
- **20:29:19 · Jonathan Jackson:** So the limitations here are that it can… It can inspect code, it can run as an agent, but it can't build and run, so it would… prospectively submit a pull request, and then CIA would tell you whether it did the thing or not.
- **20:29:32 · SS:** Yeah, the idea that I'm thinking here is that, yeah, like PRs would go over back to the monorepo. This thing would open PRs against the monorepo. And then CI would do its thing over there. Yeah.
- **20:29:38 · Jonathan Jackson:** Right. Yeah, and if we needed to inspect it from there, we could pull a branch and do whatever else we needed.
- **20:29:46 · SS:** Yeah.
- **20:29:47 · Jonathan Jackson:** Okay, cool. Definitely sounds like a good step, even if you're not trying to solve everything all at once, which is reasonable. You're the next topic as well, Sam.
- **20:30:00 · SS:** Oh, is this the demo? Probably should have just done this all at once. Yeah. So one of the things that I.
- **20:30:04 · Jonathan Jackson:** Oh, before you… sorry, if you're doing demos, before we get into demos real quick… Uh, we had a note here that we were gonna actually record demos. Um, but we haven't ever done that. Mostly people have opted out of wanting to be recorded, so I wanted to just check in since we have 3 demos here. do we want to record these? Should we rely on the… just the transcript summaries at this point, and whatever tasks people have written? I'm kind of okay either way. Um… I don't know. I don't know that we've ever recorded a demo.
- **20:30:35 · Pete Smith:** Alessandro. Alessandro is waiting for his Wednesday morning. Won't be the same if we don't record our demo.
- **20:30:39 · Jonathan Jackson:** Right. That's the question, right? I know, yeah, it's like this is sort of a hard way to do this. So, Sam, would you like me to record or not?
- **20:30:51 · SS:** I mean, I've already shown this to Alessandro, so I don't think he cares. Yeah. Up to you. I'm good either way. I remember. Don't worry. This is a super quick one anyway, just to show the initial problem that I'm trying to solve with… with a repo like this. Let me share my screen. Um, so I have, for a while, wanted the other half of the flaky tests project. Right now, we have a flaky test project, which keeps track of, uh, you know, tests that are failing. Um… I have traditionally been more interested in the problem of what are the workflows that are failing, uh, and why, because these things were just, like… this stuff comes up a lot, like, during Xcode upgrades in particular, so… This is something that is being used by Claude to pull the monorepo, get the actions, go and look at the code, figure out what's causing a problem, and report back and give a little write-up about what's going on. Um, the idea being that eventually you could then move something into this ready-to-fix section, and the agent would then go and actually open the PR later. Um, you know, this is just moving a… moving an automation that I've had on my machine is like a daily, like, 10 AM workflow for a while. and just putting it remotely so I don't have to think about it anymore. Which is the ultimate goal. So yeah, this is the kind of thing that I'm trying to do with this repo. Um… As an aside, OT is kind of working on… something along this as well, which I learned about as I was setting up this repo. They are setting up this template, which lives on Dub, for what it's worth, for these kind of analyzing report workflows. Umm. So that might be interesting if you are needing like access from things within the system already. Uh, like, I think Lucas is currently looking at getting that thing to have sentry access and all this kind of stuff, so… Yeah. Go check that project out if you're interested. That's it.
- **20:32:48 · Jonathan Jackson:** Awesome. Thank you. Pete, you're next.
- **20:32:57 · Pete Smith:** Yeah, thanks. Yeah, this is in the demo section. It's not really a demo, I guess, but just wanted to share. I'll speak a little bit about, um. ASTAC, so I put a link in… a link into the chat there, so I'm sure people have heard… of… PSTAC, or maybe people have heard of PSTAC. I'm going to just share my screen. So if you haven't, um… Pstack is a plugin created by an engineer at Cursor or SpaceX. It's basically just a set of skills that this engineer found were very good. at, um, fixing bugs, implementing features. So they created the PSTACK plugin. What ASTACK is, is basically just a port of PSTACK. So PSTACK was. built for cursor. So it has a lot of built in you know, instructions in terms of cursor models and how they work so forth. So I just. Ported PSTAC, uh, over… To, uh, ASTAC, um… Basically replacing any of the cursor references with Cloud Code and Codex references. So what AStack is, is a plugin that you can use in Codex or Cloud Code. It's like it sounds very abstract. It's not. It's really just a. A set of skills, a set of playbooks, some defined principles. And they all work together to what I found to just to produce good output. um… So I'll just, I'll talk a little bit in a moment about how ASTAG works. So how it's set up at the moment, you can see on the left, I just have it there under my public, under my own public repo. But if anybody wants to try it out, I would encourage it. You can see like I didn't do a lot of work here. I really want to undersell this. asked an agent to port this over. So take PStack, port it, make sure it works with Cloud Code and Codex. The most work I did was creating that logo you see on the left of the screen, so that's where all my creativity went into. Um… But yeah, you can go here, you can grab the plugin, you can install it in Cloud Code. And you can try it out. I'd recommend trying it out for… for bug fixes, even like small features that are well defined. So, you know, the evidence, like any evidence I have is just anecdotal. I've just found these particular set of skills, these set of playbooks and the principles that are defined just produce very good output. Again, anecdotal. I haven't measured this. I also think it seems to do it at lower token usage. But again, I haven't measured it. So how it works, how ASTAC or PSTAC work is, you know, you install the plugin, it sets up a hook, so at the start of your sessions, the agents are aware that ASTAC is there. And if you say, hey, build this feature or fix this bug or whatever, you can explicitly say. Enter potato mode. But that's actually part of the hook. The agent will know, oh, if you give it a feature or a bug, it will enter potato mode. All that is like a top level skill, which is basically a router. And depending on the task that you give it, like fix this bug or implement this feature. You will go and find the right playbook and follow that playbook and in the playbook. That would reference what skills to use, et cetera. So. again, you know, there's nothing, like, there's nothing magical here. I mean, there's been other plugins that have been very popular. I was talking to Fernando earlier, and we talked about… was it the Superpowers one? I think was another popular plugin. So things like this will probably be, as I was speaking to Fernando about, they'll probably be, can, you know, eventually end up in the harness. Probably won't need plugins like this at some point. But at the moment, this is something I'm using daily. And I find it very beneficial. It just results in really good output. So I would encourage people to try it out. Like, what what, you know, we've seen probably saw the post from Sebastian, from OM. He's creating, like, auto, which is their version of pstack. He just took pstack. And ported that and kind of made it very specific to Duck AI. That's like the the OM version of Pete's group. So if ASTAC, if people find it useful, we could basically turn that into like the Apple version of PSTAC, Apple team version of PSTAC. and… that's something we could do. So yeah, I would just encourage people to try it out, if you're interested. There's an Asana task there that I posted, and you can give any feedback, and if people want to. People find it useful and want to contribute and make it better. We can do that. It's under AI enablement AOR. I spoke for a long time, if anybody has. Sorry, I shared that artifact. Sorry, key. I didn't go through all this artifact.
- **20:37:59 · Pete Smith:** It's linked in… The Apple Weekly Task, so you can go and look at that artifact, it explains at a very high level how it works. Again, it's not magic, it's just a skill, a ruler, lots of other skills. Sorry, my dog is barking. I don't know if you can hear my dog, but. He's over there barking at me. That's very low. That's it. If anybody has any questions, you can ask me questions, or you can ask me questions later. Jane, sorry, your hand up.
- **20:38:35 · JF:** Yeah, sorry if you said already and I missed it. What changes have you made over the original piece, Doug?
- **20:38:42 · Pete Smith:** Yeah, so… Oh, I'm sorry, it looks like. Okay, dog went dead. Um, yeah, that's why I wanted to undersell it. Like, I didn't, I didn't, I didn't do anything. So, uh, I created that logo. What, what, what I did, or what the agent did, was just port it over, support Pstack over, um, for Cloud Code and Codex. Like, the original Pstack has references to Cursor, and how Cursor. Works. So I just ported it over, so it works in… Um, codecs and cloud code. I didn't add anything additional yet. Um… But the idea will be we can add things if we find them useful. We can add Apple-specific things. If we find them useful. So yeah, no changes really. Sorry, one more thing, James. Thanks for reminding me. I'm going to reshare. Sorry, everybody. Very annoying. I'm going to reshare. So a good question. Like, PStack is being continuously updated. So I have a scheduled task that runs every Monday, looks at the latest PStack, and consumes the latest changes, and, like, ports over the latest changes. So I keep it up to date. So I look at the upstream and keep. a stack up to date. But that's other than that, I just I just port it. I don't make any other changes. Like, you know, of course, there's a chance it's not ported well. It might not work properly, but the intention is it's just a direct port.
- **20:40:09 · Jonathan Jackson:** Cool. Thank you, Pete. Paul, I'm protesting.
- **20:40:16 · Pau Ballart:** Yes, thank you. I'm going to share my screen. This is pretty much. a demo that I owe you of the snapshot testing. So, quick reminder, this is already available on iOS, not yet on macOS, because we are waiting for macOS 27. runners on big drives on the CI. on, uh… Yeah, pretty much. It's a package that has a very nice README explaining of it, what I will show today. So you can read it or tell your favorite agent to read it. But it has like requirements, how to use it. A bunch of examples for both iOS and macOS, even though, as I said, macOS. is disabled. But let's do a bit of an example. if I find the Xcode that just crashed, apparently. Ah, it's here, yeah. So, uh, how does it work? We start with the bill. Maybe, or maybe not, you will have previews. That is the new, like, hashtag… preview macro but for a snapshot so I created this one you don't need to use previews but it's just if you use them it becomes handy so there is this new preview provider that's the system. protocol, then if you conform it like this, so pretty much. You configure the different states that you want to have in the view. In this example, it's this segmented control. So there are two previews, one for each tab, let's say. You configure it here. So that's the model or the state of the view. And then you configure it here. So it will render one of the previews for each tab in this case. That basically gives you the previews here, but what is nice about it is that then you have the snapshot test. becomes basically a one liner or yeah one four liner uh but yeah just one call because it reuses the same previews that you refine it and then uh this is just a unit test like. Any other that we have. that runs the test. And this. Basically, you can configure it in different ways, for example. Here, if you want, um… to have a preview, but you don't want to snapshot it, because it's not needed. You have different. uh parameters here in this um. Let me show you. In these configurations, so there is… a scope where you can say, I want just for previews, just for snapshots, or for all of them. And then, um… in the test, you can snapshot in a different set of, um, configurations. So, in these asset snapshots. But everything is here in this package. We have different things. We have the different strategies. If you want to snapshot, for example, iPhones, iPads, all the combinations, just a single screen or a custom. Uh, size? And then you have also. the different sizes of components that you can have in an app, right? Majority of things are the screen, but you can also test some intrinsic content size, bottom sheets, or just any big size that you want. So all of that has, like, default parameters. In this example, for example, it's just, you know, normal screen, and I want to judge a snapshot in that mode, because that screen doesn't have light mode. So, it would work like this. And this is a unit test, it runs super fast, and uh… It passes in less than a second. how it works when you change something so let's say remember we have this previous let's say now i change this uh. this view. No, there is a design change, so we have this title here. If I run the tests, now they will fail. I will show you how it looks like in Xcode. So pretty much, issue recording, blah blah blah, does not match reference. And then, you can go here in your tests. We see the test that failed, and if you double-click, there is this gallery. That basically is showing us the reference, the failure, and the difference. So you can preview it like this. That's the original one. And then the new one that got recorded is this one with hello.
- **20:45:21 · Pau Ballart:** So you can see a diff that it's generated. Basically, you see everything is moving down. Um… So here it can happen two things. It's a real failure and you need to fix your UI, or it's a design change that is… expected, and you want to re-record the test flight, because this is now the new correct design. To do it, you have different ways. Here, there is a record Boolean. You can pass it to true. But at the same time, instead of changing code. If you go to the folder of the test. In in in Git you can let me show you. in a… in another screen. You can just delete the reference image in the file system. So that's the submodule that we have. In the, um… With all the snapshots. So if you go there and delete the file, it will recreate it on top of it. Um, so yeah, basically you have these two options. If there is no original file in the file system, it will just create it right away. If there is an existing one, it will do the comparison and so on. So the easiest usually is, um, to just remove. the image and re-record it. But let's say that… This works, and you upload a PR, and it fails in the CI. So how does it look like? Pretty much the current PR checks that we have for iOS, we'll report them here with the faillet, and it generates an HTML report, which is this one. that, uh… creates the same thing that we see in Xcode, but in the CI. So if you see a PR failing, you don't need to download the artifacts and so on. You just have to open this, and it has, like, these drop-downs. with the same concept, like the reference, the failure, and the diff. Um, so that's… A bit what I wanted to show. Then, um… So the snapshots live in a submodule, as you probably know. So it's a different repo. It's these DuckDuckGo Apple browser snapshots. And it just contains the PNG files. So that is this. Orchestration, if you want to run when you create snapshots, you need to open, let's say, two PRs. But there is a script that handles everything. So what you would do is here when you record your new snapshots. Maybe I can do it really quick. Just a second. I will. Show you exactly what I mean. So we have the submodule. Snapshot references. I'm gonna remove… The, um… Tests for this one. which is the scan QR code test. So I remove it from the file system. Okay, from this folder. And then when I record it or when I run the test, it will record. So first time it fails, but now it says no reference found on disk, which is fine. And then second time it runs, it already has the references and it passes. Okay, so now we want to open the PR, uh, and we can see that the snapshot reference module has the changes. Again, old and new with a new title. So there is a script in the scripts folder. around doo doo doo scripts. called Open Snapshot Zoom Module PR. You just run this over there, runs it, and it will open the PR, so commit, push, open PR, and so on. Um, and then once in your main branch or main PR, there is a new label that you can tag. Uh, let me show you here. Merge the snapshots. So this will merge the snapshots PR and update the pointer and so on. So, it's these two extra steps. You can do it manually, but, uh, the easiest is once you record it, and you have the changes here in Git, you just run the OpenPR, uh… script, and then once your main PR is approved, you can, uh, just use this label to merge the… the snapshots PR. So you…
- **20:50:21 · Pau Ballart:** It's a bit of extra work, but I try to minimize it and automate as much as possible, so… That's how we currently have it set up with these failures and so on. Any questions so far?
- **20:50:37 · Dominik Kapusta:** Yes, this is amazing, and do we have a documentation for that in the Apple documentation project?
- **20:50:44 · Pau Ballart:** For the CAI part, you mean?
- **20:50:46 · Dominik Kapusta:** Yeah, for…
- **20:50:47 · Pau Ballart:** Or in general.
- **20:50:48 · Dominik Kapusta:** In general. You've mentioned that there's a script that simplifies the work, and that would be great to have it as a task in the Apple Documentations project, if it's not there yet.
- **20:51:00 · Pau Ballart:** Okay, I will add it, yeah. Yeah, so the biggest documentation right now is the README, but I understand that it's not, uh…
- **20:51:02 · Dominik Kapusta:** Awesome. Thank you so much.
- **20:51:09 · Pau Ballart:** in the best place. I mean, it's useful, but…
- **20:51:10 · Dominik Kapusta:** Mm-hmm.
- **20:51:12 · Pau Ballart:** Yeah, I will let it also in Asana.
- **20:51:16 · Dominik Kapusta:** Thanks so much.
- **20:51:16 · Jonathan Jackson:** Very cool, thank you, Pao.
- **20:51:20 · Pau Ballart:** I have one more thing to share, which is… So two things. Why is this useful, right? Like, because, okay, we can add a snapshot test and so on. There are three things that I see that are important. First one is PR reviews. Since now the two PRs are linked, it makes it easier to review changes, right? You read UI code of SwiftUI, for example, and you see paddings, frames, blah, blah, blah. Seeing a screenshot helps a lot, so now it would be easy to compare side by side, because you will have here, um… For example, this one. The… the file changes. Directly in the… well, it's not the… This one needs a rename. And you will have the file changes on GitHub to review side by side like this. So PR reviews are better than. I think it was already mentioned we can. think about our UI tests, and if we have UI tests that are asserting. Just how things look like. It's a good candidate to replace with. with the snapshots. Um, so… I will open a proposal at some point, but the idea is… Since UI tests are flaky and snapshots and units. are less flaky and run quite fast. We might look at opportunities to move some of these tests and with a good combination of unit and snapshots. Yes, we are still missing the navigation part of I tap this button and this other screen opens. But yeah, it's up for discussion, but I think you have already a good coverage of a feature, right? And the last thing is ship reviews, because here in this other repo, it's becoming basically like an S3 bucket of data. PNG files. So when you prepare your ship review, if you have used this, you don't need, uh… to regenerate the snapshots and so on. You would just even link it from here or pick them from here. And that's just a starting point. We can improve it with future ideas, like we were talking about different languages. So how we could snapshot test different languages, different dynamic sizes for accessibility, like a large text size and so on. Even, I don't know, when the snapshots PR is open. with some automation, check the original Figma design, and compare it, and there are, there are, um… ways to improve this. So this is just a starting point. And last thing I want to mention, it was part of the tech design to track adoption of this. So far, I'm the only one contributing, which is why I'm doing this demo as well. But I created with Claude an automation that every week checks the repo. It's very easy, because we can just go to the insides of this, um… of this repo and see contributors, but also just check, uh, the asset and snapshot, uh, usages. So this is a dashboard that I shared, uh, within DuckDuckGo. It's updated every week, and basically we can see how. how it grows over time. Um… So yeah, if you fancy, please use it. And if there are any issues, let me know. I'm like the unofficial or official DRI of this. Happy to be on top of it if it's causing issues as well.
- **20:54:59 · Jonathan Jackson:** It's really cool, Pat. Thank you for sharing. Yes.
- **20:54:59 · DR:** Good work, pal.
- **20:55:04 · Jonathan Jackson:** Very cool. We are now at 5 minutes to the top of the hour. Dominic and Alex, you both have postmortems on the list. Do you want to… run those now, understanding we'll go past time, and anybody who can stay is welcome to stay or not? Or would you rather find another time for those?
- **20:55:24 · Dominik Kapusta:** I'm fine running it. It should also be short, but not sure about Alexa's one.
- **20:55:33 · Alex Martemyanov:** Probably can wait till the next Apple Weekly.
- **20:55:37 · Jonathan Jackson:** Okay.
- **20:55:37 · Alex Martemyanov:** I think it was waiting for a year and it didn't wait another week.
- **20:55:41 · Jonathan Jackson:** All right.
- **20:55:42 · Alex Martemyanov:** Thank you.
- **20:55:43 · Jonathan Jackson:** Thank you, Alex. Um, then, Dominic, why don't you go ahead and get started? Anybody who's not interested in Dominic's incident post-mortem, obviously, is welcome to drop. If we run a little bit late and you need to be somewhere else, that's also okay to drop. And Alex, I'll keep yours on the agenda for next week. Thank you.
- **20:56:01 · Dominik Kapusta:** Thanks. I've dropped the link in the agenda as well for the incident postmortem. What happened last Monday is we released the app that was crashing on macOS 12. And, uh, the app was out… well, uh, so the app was out for, like. 3 hours before we, uh, realized that there is a crashing scenario on macOS 12, and then that was handled real quick, uh, with app store rollout stopped. And, uh, Sparkle rollout stopped, uh, as well at some point, um, and the fix was released, actually… well, it… It. Well, almost immediately on DMG, and the next day on App Store, because of the review time. Um, but, uh, how it happened? So, it happened, uh, because we had a change to, um, the save credentials pop-up, which shows up every time you're offered to… you're being offered to save a password. Um… On macOS 12, um… And that change was due to dropping a storyboard, replacing a storyboard with a code-initialized view. So what didn't go as planned? This, uh, change was not tested on old OS versions before merging, and it was strictly macOS 12 issue. Uh, like I mentioned before in this call, we had that engineering expectation in place, but we're not really enforcing it for every change. There's no way of enforcing it, uh, and there's no way of. testing, um… all those versions in CI, so… that's why it happened. Really. Uh, the other thing is, uh, we didn't know about the crash, because, uh, I overlooked it in Sentry. I was the release DRA and I overlooked it in Sentry. There was a single crash on macOS 12 there. Uh, that I treated as noise, and I went ahead and released the app. The sentry summaries weren't generated on that day, because we had another… Whoopsie. Um, the week before, when we updated, uh. Well, some updated the model version and I reviewed it and approved it. and we merged that change, and we updated the model version, but we didn't update the SDK version, and the old SDK didn't… wasn't recognizing the new model, and it failed to… collaborate, so we didn't generate summaries. Um… So that's the other… the other thing. Uh, when the… but! When the summary was generated, uh, it clearly pointed out that there is a crashing scenario on Mac OS 12, which started the incident. A week of internal testing didn't catch a relatively easy-to-reproduce crash. It was a 100% crashing scenario, which wasn't caught. Because we don't have testers on macOS 12. Well, apparently we have one, because we got this one crash at some point. I don't know who it is. I'm…
- **20:59:33 · Alex Martemyanov:** Yeah, and by the way, there was another kind of related issue about the crash on Intel machines. It's not directly related, but yeah, it was close.
- **20:59:41 · Dominik Kapusta:** Yes, another one, exactly. We also don't have testers, or many testers, on x86. architecture. The App Store review took 15 hours, which is rather long for an expedited review, but also there's nothing we can do about it. At least it got in the queue… I mean, it got in the review pretty quickly after requesting a completed review. What went well is that the sentry summary that was eventually generated, um… Clearly identified the crash, and suggested addressing it, and the fix was simple, quick to prepare and test, it was just assigning an initial frame to a view. And the hotfix release automation workflow less at this time. It's always important to point this out, because… We only talk about CI when it's not working correctly. Despite the long review time, we only had 37 crashes in total and only 9 affected users. That's all. for action items and DRIs, we actually, uh… there was an action item to update agent's instruction prefect… essentially update prefect to ensure that Claude agent SDK version is… compatible with used models. Uh, Sam has handled it, and, uh, well, there is a code command right now to, to update the SDK version whenever you update the model version. Um… The other topic that's not actually mentioned here is what to do about testing on older OS versions, but this is something we will be handling separately through Alex's work, mentioned earlier on the call. And… That's it. Finally, I created an incident task through the form. Asana forum, and marked the incident as high, because it's a crashing scenario, we need a hotfix, and an SRE person joined the call, and we talked a little bit, and he, um… Explain to me that if it's a one-person rollout. It's not a high severity incident, and we dropped it to medium, and they dropped the call. And that's it. And, um… That's something to keep in mind for later, for next incidents, when they are only starting rolling out, we may not need SRA help there. This is all I have.
- **21:02:09 · SS:** Regarding the action item, are you happy enough with just having the code comment now that reminds about the Claude SDK?
- **21:02:15 · Dominik Kapusta:** I think so.
- **21:02:16 · SS:** you know, it's… it's not bulletproof, like, what I… what I wanted to do was to just ask the Claude SDK, what models do you support, and check that it's in the list, but it only includes… yeah, it doesn't include all the actual supported models.
- **21:02:18 · Dominik Kapusta:** I think, yeah. I saw that, yeah.
- **21:02:28 · SS:** Eh, maybe the comment's okay for now. I tried to make it very obvious what to do and how to confirm this.
- **21:02:32 · Dominik Kapusta:** Yeah, I mean, that's okay. Also, that comment would show up, because it's just very close to the line that is changing, it will show up in the GitHub UI when reviewing the code, so it should be fine.
- **21:02:37 · SS:** It's right there. Yeah, the only way I was considering extending this was to just have a GitHub Actions check that looks for model changes and adds a comment, GitHub Actions comments. But I would want to do this for all of Prefect. Whatever. I'll look at that.
- **21:02:56 · Dominik Kapusta:** Yeah, yeah. Should be fine. Okay.
- **21:02:58 · SS:** Okay.
- **21:03:00 · Dominik Kapusta:** Thank you. Thanks for handling that, Sam. Alex?
- **21:03:05 · Alex Martemyanov:** Yeah, I got a proposal in… I got it related to post-mortem of the VS Code, but I think it more suits here. What about adding… updating our process to require adding UI tests in addition to renting some older Mac to run older OSs, but make a requirement to the process to cover any UI change. with a UI test. Even if it's a fix or a quick addition in the uncovered region, especially having all the Agentic workflows, it may be a quick and cheap option. I mean, cheap to add. maybe not to maintain. But it adds, of course, it adds extra cost for the runners. Or maybe we could have like a test suite number two that runs weekly, for example, and not daily.
- **21:04:11 · Dominik Kapusta:** I actually think it's… Some Windows team uses something similar. whenever they change UI, they add UI tests, and they have plenty of UI tests, and that's another problem. I believe, but… Yeah, we could… We could discuss it, I guess.
- **21:04:31 · Alex Martemyanov:** Yeah, because, uh…
- **21:04:31 · Jonathan Jackson:** I think that's what we're considering. Sorry, go ahead.
- **21:04:35 · Alex Martemyanov:** Because obvious, like, obvious… thing that could fix it is, uh, like, QA, which we are missing, for… everybody knows for… for why, for what reasons, uh, but we have a lot of, uh, supported platforms, a lot of supported macOS versions, and, uh, it's, uh… Like, uh, it's impossible to… make the requirement as in the mentioned. Uh, process, like, to test all the supported macOS versions, because not everybody even have all these machines that, like, I have the older one, but it's not DuckDuckGo, it's my personal machine. And, uh, I can test it, but it takes time. Uh, and nobody… nobody does it, frankly. And we have no QA, and it's impossible to guarantee the quality in such conditions. So, maybe it's required. It should be a required condition to… to… raise the UI test coverage, the end-to-end coverage, so we can validate we are actually producing quality UI tests. Stuff.
- **21:05:50 · Jonathan Jackson:** Yeah, I think that's worth considering alongside the virtual machine testing stuff that you were talking about earlier, right? I mean, we've known for a long time that we clearly have testing gaps, especially on older versions and older hardware. And… To this point, I think we've just accepted that as the way it is and every now and then something like this goes wrong and we deal with it. um… But we should definitely think about whether there's a promise we're making to our users there that we're not able to fulfill properly. And I think certainly that's true right now. The question is, what does it take to do better at that promise? So I'd love to see some proposals around moving some of these pieces forward.
- **21:06:32 · Dominik Kapusta:** All right, that was all for me. Thank you very much.
- **21:06:38 · Jonathan Jackson:** Alright, and thank you. Um, so, Alex, we're gonna wait till next week for your other postmortem. Thank you for your flexibility there. And thank you all who stayed to finish that out. I appreciate it. And I look forward to seeing you again soon. Bye, everyone.
- **21:06:54 · Alex Martemyanov:** Thank you. Bye.
+30 -14
View File
@@ -86,56 +86,72 @@
"meeting_date": null,
"source_url": "https://us01mynotes.zoom.us/rJFn9D3lRJmnlCMSomhPsw",
"folder": "undated Michal Smaga's Personal Meeting Room rJFn9D3lRJmnlCMSomhPsw",
"sha256": "1fae59c1ab37063ae38b62d3b8da4c44c98ee90e7970e00a04332e589971366d",
"cues": 3
"sha256": "ddc1d2b363aac0ac95d37b5c3bc36774066f85625d7dc00096eb6d652a849ab7",
"cues": 60
},
"DLwAbUQ_RZ6X_D63xiX_Ww": {
"title": "Max Tsoy's Personal Meeting Room",
"meeting_date": null,
"source_url": "https://us01mynotes.zoom.us/DLwAbUQ_RZ6X_D63xiX_Ww",
"folder": "undated Max Tsoy's Personal Meeting Room DLwAbUQ_RZ6X_D63xiX_Ww",
"sha256": "30876dc0e175096e163ba24f8a2f89e1653aef009ee7a48b6498f3ff29700b3b",
"cues": 3
"sha256": "e5fa0a90b94a69b823c6c97e5147fb9eae0a91614b3bad1933b67f4f29784979",
"cues": 115
},
"VSuzeqZCR5O2z5KtmaQt3g": {
"title": "Apple Weekly Team Meeting - Zoom ID",
"meeting_date": null,
"source_url": "https://us01mynotes.zoom.us/VSuzeqZCR5O2z5KtmaQt3g",
"folder": "undated Apple Weekly Team Meeting - Zoom ID VSuzeqZCR5O2z5KtmaQt3g",
"sha256": "5fb625f0ac5196072b0aaa3090f0145b6f2d038b221884ddbe73c4a42c1600a5",
"cues": 3
"sha256": "1a1578041bd99d565ced6dd0b7fdaeb7deb6a2126d7ebad85d2f3376daa47c92",
"cues": 38
},
"p5FdcX9WT-afBpWjRgaW0w": {
"title": "Dominik Kapusta's Personal Meeting Room",
"meeting_date": null,
"source_url": "https://us01mynotes.zoom.us/p5FdcX9WT-afBpWjRgaW0w",
"folder": "undated Dominik Kapusta's Personal Meeting Room p5FdcX9WT-afBpWjRgaW0w",
"sha256": "3e71412712b610ff9d470bce9570191a2c17f95bf6b8509ac2d7b4764f166f89",
"cues": 3
"sha256": "a99be6b590cec73dfa0e9d9c09f85745070e294ef2fe2dd3c5178a56479ca8f4",
"cues": 130
},
"e5TYWRytRz2WwWSe68BSWQ": {
"title": "O-E 2026",
"meeting_date": null,
"source_url": "https://us01mynotes.zoom.us/e5TYWRytRz2WwWSe68BSWQ",
"folder": "undated O-E 2026 e5TYWRytRz2WwWSe68BSWQ",
"sha256": "416b676d9aaf403d9495351885a0def9a9508acf3571ccd44948940d7e6f4df4",
"cues": 3
"sha256": "fb1dd73affd413ab4f07eeeef61929d3acff8b33e0727a00e710677bc605954d",
"cues": 107
},
"3sv2crbiShun--0lAywwvg": {
"title": "Alex Martemyanov's Zoom Meeting",
"meeting_date": null,
"source_url": "https://us01mynotes.zoom.us/3sv2crbiShun--0lAywwvg",
"folder": "undated Alex Martemyanov's Zoom Meeting 3sv2crbiShun--0lAywwvg",
"sha256": "9bc58352ca8f1c475195e58dd9fa208200aaddbdef95b04e32110e9d9a47a9c7",
"cues": 3
"sha256": "e7548ab7bd46e90a26cf4983bdd90f34388fa977b5f91b8516324a2821d9303f",
"cues": 80
},
"gBBa-iiUQLycw55WRV1tnw": {
"title": "Apple Weekly Team Meeting - Zoom ID 2026-10-20 20:01(GMT+6:00)",
"meeting_date": "2026-10-20",
"source_url": "https://us01mynotes.zoom.us/gBBa-iiUQLycw55WRV1tnw",
"folder": "2026-10-20 Apple Weekly Team Meeting - Zoom ID 2026-10-20 20 01(GMT+6 00) gBBa-iiUQLycw55WRV1tnw",
"sha256": "faa9a63cee2b3ef985a5edfb122b79b5e604ab37bc1fafff519f5e6858ea2872",
"cues": 3
"sha256": "ca11ac2c5c991594f880644e97af7b6e66458623484464f46482e2e7578f7832",
"cues": 137
},
"7JZgnceAT1aZ2H75ok4dmA": {
"title": "Costas Brammer's Personal Meeting Room",
"meeting_date": null,
"source_url": "https://us01mynotes.zoom.us/7JZgnceAT1aZ2H75ok4dmA",
"folder": "undated Costas Brammer's Personal Meeting Room 7JZgnceAT1aZ2H75ok4dmA",
"sha256": "0542896987d39748e9a796d9a90a55f33d3432ff3697b2d14461e7121eee6282",
"cues": 23
},
"bPprGg5DTye9H-RQXkwMuw": {
"title": "All-Company Meeting",
"meeting_date": null,
"source_url": "https://us01mynotes.zoom.us/bPprGg5DTye9H-RQXkwMuw",
"folder": "undated All-Company Meeting bPprGg5DTye9H-RQXkwMuw",
"sha256": "3535bde61224da638a110e4e65177e18b50bce689afd61b21a87fd2b87a2da94",
"cues": 50
}
}
}
@@ -19,6 +19,391 @@
"time": "16:15:40",
"speaker": "Michal Smaga",
"text": "Uh, I wanted to, like, quickly think about the next steps for the CPM project, and… Uh, so, from my understanding, like, and I agree with that, like, we are focusing on macOS, like, mostly, and trying to resolve the… The permanent, uh… Issues, like, with the broken messaging."
},
{
"time": "16:16:04",
"speaker": "Alex Martemyanov",
"text": "Yes, so it's mostly pharmacoeleus and permanent, but they generally have. They are intersecting."
},
{
"time": "16:16:11",
"speaker": "Michal Smaga",
"text": "Mmhm."
},
{
"time": "16:16:13",
"speaker": "Alex Martemyanov",
"text": "Um, and uh…"
},
{
"time": "16:16:16",
"speaker": "Michal Smaga",
"text": "So, from the…"
},
{
"time": "16:16:16",
"speaker": "Alex Martemyanov",
"text": "Yeah, so the state is…"
},
{
"time": "16:16:19",
"speaker": "Michal Smaga",
"text": "So we have, like, I think, 2 or 3 known…"
},
{
"time": "16:16:19",
"speaker": "Alex Martemyanov",
"text": "Good."
},
{
"time": "16:16:22",
"speaker": "Michal Smaga",
"text": "Uh, which is, like, the…"
},
{
"time": "16:16:23",
"speaker": "Alex Martemyanov",
"text": "Yes, we have…"
},
{
"time": "16:16:25",
"speaker": "Michal Smaga",
"text": "The, the, um… The feature flag removal, but it's, like, it will be done. Then we have the update, extension update path."
},
{
"time": "16:16:38",
"speaker": "Alex Martemyanov",
"text": "Thank you. Kind of. So, first of all, we have, uh, the P3 base URL fix that's going on Monday, and it's a significant part. Uh, then it's, uh… Error number six at the time currently. Opening pull request 4. It's basically reproduced in iOS. Uh, when… so basically when, uh… Background view is suspended. It's slowed. Prepare this sync. Signal, uh… signal suspend, signal resume. So how iOS suspends up when the home screen is shown. If we are in the middle of the request, it basically hangs and we need to reload the context to… to make it work again. And it seems the fixes are working for it. So it's first reproduced and fixed. There is this idle state that is kind of unknown. And, uh, there is…"
},
{
"time": "16:17:52",
"speaker": "Michal Smaga",
"text": "Uh, one question, one question for the idle state. Do you think, like, because. Uh, I know that the, the, like… the events that we log in, like, can be, like, very long. So do you think it may be… other kind of issue, but we, we, like, kind of misreported, or, like, we, like, the origin is getting closed. Because the session is so long."
},
{
"time": "16:18:23",
"speaker": "Alex Martemyanov",
"text": "is… So first of all, it was kind of P1 in the original. fix much. I'm here. So it's… Small idle weight. Mm-hmm. So basically a timeout. Time out a lot. And, uh… So, currently, there is no more info on this, and no steps. No steps to reproduce. Mmhm. It seems mostly like a regular state, and the… There is some suspension mentioned probably because of the long sub life cycle. And it's generally pointing to a regular life cycle over time. So it's kinda… Says nothing, actually. And this is the biggest group."
},
{
"time": "16:19:28",
"speaker": "Michal Smaga",
"text": "Yeah, but what I mean is, like, if we, like… Do you think it can be connected with other issues? Like, for example, the the base the base URL, like, for example, maybe fixing base URL."
},
{
"time": "16:19:39",
"speaker": "Alex Martemyanov",
"text": "Hmm, what's… Maybe."
},
{
"time": "16:19:45",
"speaker": "Michal Smaga",
"text": "May may, like, as a side effect, fix the the the idle, like, as well."
},
{
"time": "16:19:50",
"speaker": "Alex Martemyanov",
"text": "Well, maybe, but the previous fixes we've had didn't affect it. And so generally these groups are distributed because of the different footprint of the events. And, uh… So they mostly kind of don't intersect and… have a distinct grouping. And this, uh, like… Mostly shows the general DNA trend. A week from the ideal state. And it seems at some point that… One slot. Maybe it could be affected by the… And on base URL… Maybe it's kind of related to… well, it's not related to the updated extension, because they didn't have… the extension update parameter. We read it, uh, wait time. A timeout. Five second wait. It didn't help, so it's… Mostly, like, some permanent failure. So, frankly, no ideas at the moment. Just…"
},
{
"time": "16:21:15",
"speaker": "Michal Smaga",
"text": "So, so you have the, the fix upcoming for the error 6, right?"
},
{
"time": "16:21:21",
"speaker": "Alex Martemyanov",
"text": "Yes, uh, the reward for… for 6."
},
{
"time": "16:21:22",
"speaker": "Michal Smaga",
"text": "And… Will you be looking into the embedded extension update thing, just to figure out if there are, like, any race conditions there?"
},
{
"time": "16:21:34",
"speaker": "Alex Martemyanov",
"text": "Yes, yes, next I, uh, didn't look at it yet, so currently I'm evaluating the. Oh. The feature flag thing, that is not the feature flag thing. I mean the duplicate context or missing context. And another one is RR6. Next, I'm switching to… the idle state and the extension update."
},
{
"time": "16:22:00",
"speaker": "Michal Smaga",
"text": "Yeah, I think you can ignore the future flag thing, because, like."
},
{
"time": "16:22:01",
"speaker": "Alex Martemyanov",
"text": "Salish."
},
{
"time": "16:22:05",
"speaker": "Michal Smaga",
"text": "It will be merging the PR today, so, like, we can observe the stats next week, so…"
},
{
"time": "16:22:05",
"speaker": "Alex Martemyanov",
"text": "It's not… It's not the feature flag, I can tell you, because it's not being toggled, but it's…"
},
{
"time": "16:22:13",
"speaker": "Michal Smaga",
"text": "Yeah. Yeah, exactly."
},
{
"time": "16:22:18",
"speaker": "Alex Martemyanov",
"text": "If, uh… If we toggle the feature flag, there is a very similar pixel event sequence. That's why it's flagged as."
},
{
"time": "16:22:29",
"speaker": "Michal Smaga",
"text": "Okay."
},
{
"time": "16:22:30",
"speaker": "Alex Martemyanov",
"text": "Maybe feature flag toggle, but obviously it shouldn't be related to the actual feature flag. So, yeah, there's still… Also, there are little to say about it, and actual…"
},
{
"time": "16:22:43",
"speaker": "Michal Smaga",
"text": "And about the idle wake, there's, like, no ideas, like, how to move forward with that."
},
{
"time": "16:22:48",
"speaker": "Alex Martemyanov",
"text": "Not yet, I just postponed it at the moment, so I didn't dig into it since yesterday."
},
{
"time": "16:22:54",
"speaker": "Michal Smaga",
"text": "And… One question, because Max was saying that he is getting some reports from Gabe. Do you think if getting, like… how getting the the like up in a broken state like in a in real life. Could it help you anyhow? Like, could we dump something from…"
},
{
"time": "16:23:18",
"speaker": "Alex Martemyanov",
"text": "Okay. If there were any… Steps to reproduce. And so I've been thinking if we could maybe extend the telemetry. Make a specialized build. But yeah, firstly, we need to understand which kind of breakage is it. Thank you. So we cannot relate which group and which pixel is sent from games. machine. Right."
},
{
"time": "16:23:59",
"speaker": "Michal Smaga",
"text": "Don't we have, like, a local dump pixel logs or something like that? We we can dump logs from from the the app. Right?"
},
{
"time": "16:24:05",
"speaker": "Alex Martemyanov",
"text": "Oh… From console, from the console. But they stay there for very long. In the end of time."
},
{
"time": "16:24:19",
"speaker": "Michal Smaga",
"text": "We have export logs, and they have… there are pixels, uh…"
},
{
"time": "16:24:22",
"speaker": "Alex Martemyanov",
"text": "Yeah. Yes."
},
{
"time": "16:24:27",
"speaker": "Michal Smaga",
"text": "Uh… no. Could you write a step that Max could share, like, with Gabe, for example, like, if they, like, reproduce the thing to… because I assume, like, the… if… if the… if they encounter something that we detect with the telemetry, there is this, like, 5-minute window for… for dumping the… The episode, right?"
},
{
"time": "16:24:50",
"speaker": "Alex Martemyanov",
"text": "Mmm… yes."
},
{
"time": "16:24:51",
"speaker": "Michal Smaga",
"text": "So, getting the… maybe getting the pixel logs from the device within the time frame of that? Like, 5 minutes after encountering the thing."
},
{
"time": "16:25:01",
"speaker": "Alex Martemyanov",
"text": "5 minutes, yeah, 5 minutes after, if it's not, uh… If it's not restored, if it's not recovered. And uh… No signals."
},
{
"time": "16:25:15",
"speaker": "Michal Smaga",
"text": "Could you double-check, like, what kind of, like, uh… settings, like, if I'm checking everything, and, like, getting the pixels, and, like, basically write up a short instruction, probably, in Asana, so Max can share it with anyone?"
},
{
"time": "16:25:30",
"speaker": "Alex Martemyanov",
"text": "Okay. Okay, basically… Just exporting the pixels, I don't know, would it… VPN… Yeah, basically, we don't need VPN, and uh… Personal information removal. I think… But other stuff may be helpful, because the pixels would show the same exact info we already are getting from. the production browser. I've got some related…"
},
{
"time": "16:26:15",
"speaker": "Michal Smaga",
"text": "So…"
},
{
"time": "16:26:17",
"speaker": "Alex Martemyanov",
"text": "surrounding events may be helpful."
},
{
"time": "16:26:20",
"speaker": "Michal Smaga",
"text": "Do you know what is the update file?"
},
{
"time": "16:26:24",
"speaker": "Alex Martemyanov",
"text": "Mm, not really."
},
{
"time": "16:26:27",
"speaker": "Michal Smaga",
"text": "So, just pixels, or, like, that go as well?"
},
{
"time": "16:26:31",
"speaker": "Alex Martemyanov",
"text": "I think Del Dugo is wrong."
},
{
"time": "16:26:37",
"speaker": "Michal Smaga",
"text": "Okay, I'll reply to Max in them."
},
{
"time": "16:26:38",
"speaker": "Alex Martemyanov",
"text": "I mean, just pixels, we already have the pixels, maybe there would be some extra… Stuff happening around, but maybe… some other indicators."
},
{
"time": "16:26:49",
"speaker": "Michal Smaga",
"text": "Yeah, but… yeah, but I think, like, it would be good… Good to figure out what kind of breakage they encounter, right? Because probably this is, like, the most popular, like, one that they are running into, and, like, to get, like, to match to which category it falls in."
},
{
"time": "16:27:00",
"speaker": "Alex Martemyanov",
"text": "Okay. Yeah, so… Should I send it to you, or…"
},
{
"time": "16:27:19",
"speaker": "Michal Smaga",
"text": "Uh, I'll write up a message for Max, like, I will reply for the…"
},
{
"time": "16:27:24",
"speaker": "Alex Martemyanov",
"text": "Yeah, basically, basically exports, debug logs, logging, and export logs. And, uh, yeah, check the pixels. And I'll check with Ian and Pierre. Yeah, so I'm… Proceeding with the pull requests… I'm the one that… It's reproduced. And then we'll… We can go on."
},
{
"time": "16:28:05",
"speaker": "Michal Smaga",
"text": "So, if you have those fixes. Do you have anything else to focus on in the project?"
},
{
"time": "16:28:13",
"speaker": "Alex Martemyanov",
"text": "And… You mean CPM? If I have those fixes. Probably won't have the fixes. Earlier than this week. And…"
},
{
"time": "16:28:33",
"speaker": "Michal Smaga",
"text": "And also, we can double-check the stats, like, later in the week, right? Thursday, Friday."
},
{
"time": "16:28:33",
"speaker": "Alex Martemyanov",
"text": "And, uh… Yeah. In other words, they're still… Some major work unfinished hanging around. Yeah, basically just working on this, on the groups that we've got. I'm thinking maybe just prepare a production build."
},
{
"time": "16:28:59",
"speaker": "Michal Smaga",
"text": "Cool."
},
{
"time": "16:29:04",
"speaker": "Alex Martemyanov",
"text": "For… for myself. Adding some extra notification messages to it when we detect the hang. And, uh… Just try to use it, maybe, on a regular basis."
},
{
"time": "16:29:19",
"speaker": "Michal Smaga",
"text": "Well, one question, like…"
},
{
"time": "16:29:19",
"speaker": "Alex Martemyanov",
"text": "For everyday use and try switching."
},
{
"time": "16:29:22",
"speaker": "Michal Smaga",
"text": "Uh-huh, random idea, like… Do you think we could add something that would help us to, like, in the debug menu, to, like… Dump the current state of something, or, like, uh, maybe the, the, the, like, display the… Not to have to dig through the logs, but, like, display the current, like, episode or, like, state of the CPM health monitor or whatever."
},
{
"time": "16:29:54",
"speaker": "Alex Martemyanov",
"text": "We… we already have. the parameters. and describe whether it's loaded and. Reviewed and allocated. Other stuff."
},
{
"time": "16:30:11",
"speaker": "Michal Smaga",
"text": "Yeah, what I'm thinking is, like, if, for example, Gabe or anyone in the company, like, experiences the issue, to… like, uh… Maybe it would be interesting to figure out what kind of issue they are encountering and focus on that. the most."
},
{
"time": "16:30:35",
"speaker": "Alex Martemyanov",
"text": "It would make sense. Again, pixels should help in it. Just exploding below. So we should, so we know exactly what's happening, what kind, which group does it fall to? But yeah, other than that, I don't know. It's about where we don't know where to look, and it may be related to the… Pull up Windows. There's some… To some other kind of events of session restoration, or… Wake and tribal events. So yeah, identifying the exact case would help. But otherwise, I don't know if… where the key is hidden."
},
{
"time": "16:32:21",
"speaker": "Michal Smaga",
"text": "About deluxe, it's like five minutes from detecting the episode happening, right?"
},
{
"time": "16:32:29",
"speaker": "Alex Martemyanov",
"text": "Uh, yes. But, uh, if we have the logs… Yeah, the log should be 10 minutes, right? Yeah, let's try. Maybe it will give some links."
},
{
"time": "16:34:07",
"speaker": "Michal Smaga",
"text": "Cool. Is there anything else I can help you with?"
},
{
"time": "16:34:13",
"speaker": "Alex Martemyanov",
"text": "Mmm… Trying to give us the Snyders, maybe. Looking at the same data. Yeah, I'll ping you later on with the pull request when it's ready. So we have the reproduced tissue, at least one. And… Yeah, just going on for the… The rest of the code base may be related, evaluated, and evaluating different guesses. Whether we kind of lose anything. At any upstage."
},
{
"time": "16:35:13",
"speaker": "Michal Smaga",
"text": "Okay."
},
{
"time": "16:35:13",
"speaker": "Alex Martemyanov",
"text": "That's my plan."
},
{
"time": "16:35:16",
"speaker": "Michal Smaga",
"text": "If you need any help, or, like, we'll have the PRs, like, just feel free to ping me, I'll try to be…"
},
{
"time": "16:35:21",
"speaker": "Alex Martemyanov",
"text": "Yeah, sure, I can deal with it."
},
{
"time": "16:35:24",
"speaker": "Michal Smaga",
"text": "Quick about that. Mm-hmm. No problem. Okay, so… Good luck, and… Have a nice rest of the day. Yeah. See you."
},
{
"time": "16:35:34",
"speaker": "Alex Martemyanov",
"text": "Thanks. You too."
}
]
}
@@ -14,3 +14,80 @@ source: "https://us01mynotes.zoom.us/3sv2crbiShun--0lAywwvg"
- **16:15:37 · Michal Smaga:** Hey, Alex.
- **16:15:38 · Alex Martemyanov:** Hey.
- **16:15:40 · Michal Smaga:** Uh, I wanted to, like, quickly think about the next steps for the CPM project, and… Uh, so, from my understanding, like, and I agree with that, like, we are focusing on macOS, like, mostly, and trying to resolve the… The permanent, uh… Issues, like, with the broken messaging.
- **16:16:04 · Alex Martemyanov:** Yes, so it's mostly pharmacoeleus and permanent, but they generally have. They are intersecting.
- **16:16:11 · Michal Smaga:** Mmhm.
- **16:16:13 · Alex Martemyanov:** Um, and uh…
- **16:16:16 · Michal Smaga:** So, from the…
- **16:16:16 · Alex Martemyanov:** Yeah, so the state is…
- **16:16:19 · Michal Smaga:** So we have, like, I think, 2 or 3 known…
- **16:16:19 · Alex Martemyanov:** Good.
- **16:16:22 · Michal Smaga:** Uh, which is, like, the…
- **16:16:23 · Alex Martemyanov:** Yes, we have…
- **16:16:25 · Michal Smaga:** The, the, um… The feature flag removal, but it's, like, it will be done. Then we have the update, extension update path.
- **16:16:38 · Alex Martemyanov:** Thank you. Kind of. So, first of all, we have, uh, the P3 base URL fix that's going on Monday, and it's a significant part. Uh, then it's, uh… Error number six at the time currently. Opening pull request 4. It's basically reproduced in iOS. Uh, when… so basically when, uh… Background view is suspended. It's slowed. Prepare this sync. Signal, uh… signal suspend, signal resume. So how iOS suspends up when the home screen is shown. If we are in the middle of the request, it basically hangs and we need to reload the context to… to make it work again. And it seems the fixes are working for it. So it's first reproduced and fixed. There is this idle state that is kind of unknown. And, uh, there is…
- **16:17:52 · Michal Smaga:** Uh, one question, one question for the idle state. Do you think, like, because. Uh, I know that the, the, like… the events that we log in, like, can be, like, very long. So do you think it may be… other kind of issue, but we, we, like, kind of misreported, or, like, we, like, the origin is getting closed. Because the session is so long.
- **16:18:23 · Alex Martemyanov:** is… So first of all, it was kind of P1 in the original. fix much. I'm here. So it's… Small idle weight. Mm-hmm. So basically a timeout. Time out a lot. And, uh… So, currently, there is no more info on this, and no steps. No steps to reproduce. Mmhm. It seems mostly like a regular state, and the… There is some suspension mentioned probably because of the long sub life cycle. And it's generally pointing to a regular life cycle over time. So it's kinda… Says nothing, actually. And this is the biggest group.
- **16:19:28 · Michal Smaga:** Yeah, but what I mean is, like, if we, like… Do you think it can be connected with other issues? Like, for example, the the base the base URL, like, for example, maybe fixing base URL.
- **16:19:39 · Alex Martemyanov:** Hmm, what's… Maybe.
- **16:19:45 · Michal Smaga:** May may, like, as a side effect, fix the the the idle, like, as well.
- **16:19:50 · Alex Martemyanov:** Well, maybe, but the previous fixes we've had didn't affect it. And so generally these groups are distributed because of the different footprint of the events. And, uh… So they mostly kind of don't intersect and… have a distinct grouping. And this, uh, like… Mostly shows the general DNA trend. A week from the ideal state. And it seems at some point that… One slot. Maybe it could be affected by the… And on base URL… Maybe it's kind of related to… well, it's not related to the updated extension, because they didn't have… the extension update parameter. We read it, uh, wait time. A timeout. Five second wait. It didn't help, so it's… Mostly, like, some permanent failure. So, frankly, no ideas at the moment. Just…
- **16:21:15 · Michal Smaga:** So, so you have the, the fix upcoming for the error 6, right?
- **16:21:21 · Alex Martemyanov:** Yes, uh, the reward for… for 6.
- **16:21:22 · Michal Smaga:** And… Will you be looking into the embedded extension update thing, just to figure out if there are, like, any race conditions there?
- **16:21:34 · Alex Martemyanov:** Yes, yes, next I, uh, didn't look at it yet, so currently I'm evaluating the. Oh. The feature flag thing, that is not the feature flag thing. I mean the duplicate context or missing context. And another one is RR6. Next, I'm switching to… the idle state and the extension update.
- **16:22:00 · Michal Smaga:** Yeah, I think you can ignore the future flag thing, because, like.
- **16:22:01 · Alex Martemyanov:** Salish.
- **16:22:05 · Michal Smaga:** It will be merging the PR today, so, like, we can observe the stats next week, so…
- **16:22:05 · Alex Martemyanov:** It's not… It's not the feature flag, I can tell you, because it's not being toggled, but it's…
- **16:22:13 · Michal Smaga:** Yeah. Yeah, exactly.
- **16:22:18 · Alex Martemyanov:** If, uh… If we toggle the feature flag, there is a very similar pixel event sequence. That's why it's flagged as.
- **16:22:29 · Michal Smaga:** Okay.
- **16:22:30 · Alex Martemyanov:** Maybe feature flag toggle, but obviously it shouldn't be related to the actual feature flag. So, yeah, there's still… Also, there are little to say about it, and actual…
- **16:22:43 · Michal Smaga:** And about the idle wake, there's, like, no ideas, like, how to move forward with that.
- **16:22:48 · Alex Martemyanov:** Not yet, I just postponed it at the moment, so I didn't dig into it since yesterday.
- **16:22:54 · Michal Smaga:** And… One question, because Max was saying that he is getting some reports from Gabe. Do you think if getting, like… how getting the the like up in a broken state like in a in real life. Could it help you anyhow? Like, could we dump something from…
- **16:23:18 · Alex Martemyanov:** Okay. If there were any… Steps to reproduce. And so I've been thinking if we could maybe extend the telemetry. Make a specialized build. But yeah, firstly, we need to understand which kind of breakage is it. Thank you. So we cannot relate which group and which pixel is sent from games. machine. Right.
- **16:23:59 · Michal Smaga:** Don't we have, like, a local dump pixel logs or something like that? We we can dump logs from from the the app. Right?
- **16:24:05 · Alex Martemyanov:** Oh… From console, from the console. But they stay there for very long. In the end of time.
- **16:24:19 · Michal Smaga:** We have export logs, and they have… there are pixels, uh…
- **16:24:22 · Alex Martemyanov:** Yeah. Yes.
- **16:24:27 · Michal Smaga:** Uh… no. Could you write a step that Max could share, like, with Gabe, for example, like, if they, like, reproduce the thing to… because I assume, like, the… if… if the… if they encounter something that we detect with the telemetry, there is this, like, 5-minute window for… for dumping the… The episode, right?
- **16:24:50 · Alex Martemyanov:** Mmm… yes.
- **16:24:51 · Michal Smaga:** So, getting the… maybe getting the pixel logs from the device within the time frame of that? Like, 5 minutes after encountering the thing.
- **16:25:01 · Alex Martemyanov:** 5 minutes, yeah, 5 minutes after, if it's not, uh… If it's not restored, if it's not recovered. And uh… No signals.
- **16:25:15 · Michal Smaga:** Could you double-check, like, what kind of, like, uh… settings, like, if I'm checking everything, and, like, getting the pixels, and, like, basically write up a short instruction, probably, in Asana, so Max can share it with anyone?
- **16:25:30 · Alex Martemyanov:** Okay. Okay, basically… Just exporting the pixels, I don't know, would it… VPN… Yeah, basically, we don't need VPN, and uh… Personal information removal. I think… But other stuff may be helpful, because the pixels would show the same exact info we already are getting from. the production browser. I've got some related…
- **16:26:15 · Michal Smaga:** So…
- **16:26:17 · Alex Martemyanov:** surrounding events may be helpful.
- **16:26:20 · Michal Smaga:** Do you know what is the update file?
- **16:26:24 · Alex Martemyanov:** Mm, not really.
- **16:26:27 · Michal Smaga:** So, just pixels, or, like, that go as well?
- **16:26:31 · Alex Martemyanov:** I think Del Dugo is wrong.
- **16:26:37 · Michal Smaga:** Okay, I'll reply to Max in them.
- **16:26:38 · Alex Martemyanov:** I mean, just pixels, we already have the pixels, maybe there would be some extra… Stuff happening around, but maybe… some other indicators.
- **16:26:49 · Michal Smaga:** Yeah, but… yeah, but I think, like, it would be good… Good to figure out what kind of breakage they encounter, right? Because probably this is, like, the most popular, like, one that they are running into, and, like, to get, like, to match to which category it falls in.
- **16:27:00 · Alex Martemyanov:** Okay. Yeah, so… Should I send it to you, or…
- **16:27:19 · Michal Smaga:** Uh, I'll write up a message for Max, like, I will reply for the…
- **16:27:24 · Alex Martemyanov:** Yeah, basically, basically exports, debug logs, logging, and export logs. And, uh, yeah, check the pixels. And I'll check with Ian and Pierre. Yeah, so I'm… Proceeding with the pull requests… I'm the one that… It's reproduced. And then we'll… We can go on.
- **16:28:05 · Michal Smaga:** So, if you have those fixes. Do you have anything else to focus on in the project?
- **16:28:13 · Alex Martemyanov:** And… You mean CPM? If I have those fixes. Probably won't have the fixes. Earlier than this week. And…
- **16:28:33 · Michal Smaga:** And also, we can double-check the stats, like, later in the week, right? Thursday, Friday.
- **16:28:33 · Alex Martemyanov:** And, uh… Yeah. In other words, they're still… Some major work unfinished hanging around. Yeah, basically just working on this, on the groups that we've got. I'm thinking maybe just prepare a production build.
- **16:28:59 · Michal Smaga:** Cool.
- **16:29:04 · Alex Martemyanov:** For… for myself. Adding some extra notification messages to it when we detect the hang. And, uh… Just try to use it, maybe, on a regular basis.
- **16:29:19 · Michal Smaga:** Well, one question, like…
- **16:29:19 · Alex Martemyanov:** For everyday use and try switching.
- **16:29:22 · Michal Smaga:** Uh-huh, random idea, like… Do you think we could add something that would help us to, like, in the debug menu, to, like… Dump the current state of something, or, like, uh, maybe the, the, the, like, display the… Not to have to dig through the logs, but, like, display the current, like, episode or, like, state of the CPM health monitor or whatever.
- **16:29:54 · Alex Martemyanov:** We… we already have. the parameters. and describe whether it's loaded and. Reviewed and allocated. Other stuff.
- **16:30:11 · Michal Smaga:** Yeah, what I'm thinking is, like, if, for example, Gabe or anyone in the company, like, experiences the issue, to… like, uh… Maybe it would be interesting to figure out what kind of issue they are encountering and focus on that. the most.
- **16:30:35 · Alex Martemyanov:** It would make sense. Again, pixels should help in it. Just exploding below. So we should, so we know exactly what's happening, what kind, which group does it fall to? But yeah, other than that, I don't know. It's about where we don't know where to look, and it may be related to the… Pull up Windows. There's some… To some other kind of events of session restoration, or… Wake and tribal events. So yeah, identifying the exact case would help. But otherwise, I don't know if… where the key is hidden.
- **16:32:21 · Michal Smaga:** About deluxe, it's like five minutes from detecting the episode happening, right?
- **16:32:29 · Alex Martemyanov:** Uh, yes. But, uh, if we have the logs… Yeah, the log should be 10 minutes, right? Yeah, let's try. Maybe it will give some links.
- **16:34:07 · Michal Smaga:** Cool. Is there anything else I can help you with?
- **16:34:13 · Alex Martemyanov:** Mmm… Trying to give us the Snyders, maybe. Looking at the same data. Yeah, I'll ping you later on with the pull request when it's ready. So we have the reproduced tissue, at least one. And… Yeah, just going on for the… The rest of the code base may be related, evaluated, and evaluating different guesses. Whether we kind of lose anything. At any upstage.
- **16:35:13 · Michal Smaga:** Okay.
- **16:35:13 · Alex Martemyanov:** That's my plan.
- **16:35:16 · Michal Smaga:** If you need any help, or, like, we'll have the PRs, like, just feel free to ping me, I'll try to be…
- **16:35:21 · Alex Martemyanov:** Yeah, sure, I can deal with it.
- **16:35:24 · Michal Smaga:** Quick about that. Mm-hmm. No problem. Okay, so… Good luck, and… Have a nice rest of the day. Yeah. See you.
- **16:35:34 · Alex Martemyanov:** Thanks. You too.
@@ -0,0 +1,259 @@
{
"schema_version": 1,
"note_id": "bPprGg5DTye9H-RQXkwMuw",
"title": "All-Company Meeting",
"meeting_date": null,
"source_url": "https://us01mynotes.zoom.us/bPprGg5DTye9H-RQXkwMuw",
"transcript": [
{
"time": "21:29:37",
"speaker": "John Boileau",
"text": "And this way."
},
{
"time": "21:30:59",
"speaker": "Gabriel Weinberg",
"text": "Hello, everybody. We have a few quick updates that we're going to mainly do the Hack Day's showcase today that Julia's leading. Everyday AI Guild. That's Zac. Okay. Yeah, there it is. All right. Yeah."
},
{
"time": "21:31:17",
"speaker": "Zach",
"text": "Yep. Uh, yeah, sorry. Um, yeah, so just quick announcement there. Uh, Julio's moving to the, uh, DRI for that, um, and Mario is going to be, uh, the secondary. Um, so they've both been pretty heavily involved in, uh, that process, and we're excited for this transition."
},
{
"time": "21:31:40",
"speaker": "Josh Frank",
"text": "I believe I'm next. So hey, everybody. I'm taking a quick break from the gym where you can frequently find me to tell you about random access learning. Just put the link in the chat. We have a session Thursday next week. 12:00 PM Eastern time, I think that's US Eastern, on corporate sustainability and being carbon negative in the AI era. So please, everybody, come to that if you can."
},
{
"time": "21:32:17",
"speaker": "AC",
"text": "Everybody Ă last week we had our away mobile browser experience. Assessment our goals days a same they car our Browser mobile, browser experience from good to great in the previous cycle. We ship the android. And dialog."
},
{
"time": "21:32:55",
"speaker": "AC",
"text": "Internal the keyboard now always on the new tape. Device to device migration. We want to ship this before black friday. A ui that s case for split. Screen. And we want to what support for many bar. We are going to start also the new tupage experiment for new users. And they think next these mark."
},
{
"time": "21:34:04",
"speaker": "Mark Ingram",
"text": "Hey, everyone. I'm Mark from OL, the Desktop Browsers Objective. Some exciting news this week is Ivy is taking over as the ODRI, and I'll be moving on to the advisor role. I'll keep it short because I know we're on hack days this week, but I also, just before introing Ivy, I want to announce my new midlife crisis to everybody, which is. I decided to start learning the drums, and I can see Carl's excited face on the screen. Okay, that's my midlife crisis update. Over. Ivy, do you want to say hi?"
},
{
"time": "21:34:31",
"speaker": "I(",
"text": "Yeah, I'm just really excited to take over OL, and stay tuned for some exciting news about extensions on macOS coming really soon."
},
{
"time": "21:34:43",
"speaker": "Julia Abreu",
"text": "All right. Congrats, Ivy. That's exciting. So I'm kicking us off today. I'm going to actually share my whole screen because I'm going to be moving from one side to the other. So I have two demos today. So we have the Julia special, two demos. Um, my first demo is… since the meetup, I've been thinking about how, um… We need to, like, I've been thinking about the new objectives that we have about word of mouth and how we talk about DuckDuckGo and how, especially for ducklings, is hard to answer some of the questions that we get once you first join DuckDuckGo. So I wanted to create something that helps them answer those questions and understand our products better. So I do want to shout out Ben Smith because he created this. I just basically said the idea. Um, so this is an artifact, um, and you will see the different tabs here, so you have different questions per tab. Um, and then you have the specific questions here, which you can click on and see them in writing. Or, if you want to, you can actually listen to it as. kind of like a podcast episode. Um, so… let me know if you can actually hear this. I'm just gonna play a couple of seconds. Episode 1. The Big Picture. Last week, a friend of mine looked up, is it normal for your knee to click? Just once. For the next week, every website they visited tried to sell them knee supports, walking sticks, and one of those… So you get the gist of it. I hope you got to hear that. Um, so this is still a work in progress. We're going to be creating this content. Um, I really didn't want to… to be in audio, because I feel like if you get that in writing, it's different explaining as a human. So, this is what we're building. And then, the second thing I'm building… dang, the Zoom… The Zoom thing doesn't let me go to the other one. Okay, here. So this is the second thing that I built. This is coming from feedback from people who went on leave, which was around how to catch up with everything that had been happening since they were gone. Um, so this is a how… what do you miss? Uh, so basically, you add here the date you started your leave, and then the start of the date you return. Um, you click catch me up. Uh, it takes a couple of seconds, but it basically gives you, like, a summary of a bunch of things that happened while you were out. So, um… company-wide posts, uh, what was published, what was answered in odds and ends, so all the Q&A, which objectives are up-to-date, like, which assessment was the last thing, and what we're working on right now, uh, things that were shipped, uh, also, like, new AORs, retired AORs, so here, as you can see, like, one, um, sample of, like, the announcements. It gives you just, like, a summary of all the announcements. Um, and we also have a people tab, so, like, new hires, which teams they are in, uh, people who left, and then it also shows you, um, green bow ties, spotlights, in case you want to catch up with all the spotlights that were posted that. Um, the time you were away. I do want to add a tab for specifically functional team updates, so, like, if there's a new process or anything specific to the functional team, I'm just not sure yet where to get that information from. Um, so… working on that. If you have suggestions, or if you would like to be… try out these things, let me know. I would love the feedback. Um, and now, this is it for my demos. Now I have a demo by, um, Robert, who couldn't be here. So, also, please ping me if you can't, uh, hear this. Hello, hello, hello. I worked on three hacks this time around. I'll try to get through it all quickly. The first is to do with increasing Duck AI prompts. I thought that maybe a nice way to do that would be to add a bit of discoverability into our browser. So if you are looking at a foreign language page, then you get a suggestion to translate it. Can someone, uh, unmute themselves and just let me know if you can hear this? Because I'm, like, concerned because I can't see that. Okay, thank you."
},
{
"time": "21:38:59",
"speaker": "Anne Hildebrand",
"text": "Yes."
},
{
"time": "21:38:59",
"speaker": "Zaahir Moolla",
"text": "We can hear."
},
{
"time": "21:39:02",
"speaker": "Julia Abreu",
"text": "And, um, so yep, that sends the page content off into Duck AI and translates it. We already had this feature, but, you know, just adding the prompt up there. That works. And there's lots of… this is very exploratory. There's lots of different ideas along these lines that we could look into, such as, if it's a long article, maybe we, um, tell the user how long it'll take and prompt to summarize, or on a shopping page, we could prompt them to, um, compare. Uh, the cool thing is that these suggestions here, these three options, that's all generated by our local LLM, so it's completely private. Um, nothing's, uh, hitting our servers until you click on one of those. Um, most recipes too. And this is a cool one, if you, um, are looking at an event, you can ask DuckAI to compute the event details. And I added this, um… dynamic UI, this card at the bottom of the Doug AI answer, and it gives you a button to add it to your calendar. Uh, one cool idea is, if you're looking at a privacy page, it could offer to help the user understand what it is they're agreeing to, so very within our brand. And another cool idea, just because YouTube features always seem to perform really well for us, is we could, um, prompt the user to, uh, do things using the transcript that appears in YouTube here. So this actually extracts the transcript from this panel. And it can send it to DocAI. Okay. Hack number two is, um, fixing a personal frustration of mine. So, I'm a Raycast user, and I'm always doing things like… um, converting currencies and, like, complex queries like this. And, uh, it frustrates me to no end that, um, that doesn't work when you plug it into DuckDuckGo. And so what I did is I had Claude reverse engineer the raycast calculator, and I've plopped that into SERP, and so now you can, um, jump into SERP running on this test machine, and do complex queries like that. And there's all sorts of things you can do. Mixed currencies is kind of cool. Um, one thing I use a lot is, like. Adding different units. And there's also, like, sort of… Complex math bits that are kind of fun. And the way that I handled this in the UI is that —. The simple calculator stays if you're doing a simple expression. And same if you're just doing a regular. conversion, then we get the existing UI for that too, because it's nicer since it has the dropdowns. Okay, and my third hack is yet another frustration that I've had for a while. So here we are looking at my phone. Oops. And, um, so we've always been able to switch tabs by swiping on the address bar at the bottom there, and that is quite nice. But what we haven't been able to do is swipe up to go to the tab switcher. And so I added that, and it turns out that Bartosz had already been working on this, so he has helped me a lot with this. Thank you very much. And yeah, it's really nice that you can… it tracks your finger, so you can kind of drag out halfway. Real smooth. And Bartosz added, you can drag out halfway and swipe to the right. Those are my hacks. Thank you very much. All right, and we're not done yet. I have another one by Pablo. When people want their data on another device they don't go looking for things and sync it back up doesn't really tell them what it does. Chanel's usability insight report made that very clear. And the problem is getting bigger. Zac AI chats, uh, can easily be lost. Um, the more stuff we add to Zac AI, um, the more. things that are worth keeping, Sink and Backup will just not be enough. So, we are proposing MyTacticGo, a private account. It's private because it's from us, and it needs no personal data to sign in, so no email and no password to remember. It is encrypted, your data is backed up, unsynced, and we show what's ready to go, like 100 passwords or 244 bookmarks, so the value is very concrete. On desktop, MyTacticGo lives in the toolbar, where people actually look. It lists what will be synced, like your chats and passwords and bookmarks, but it also brings your subscription. So, this can become, actually, your home base, and it shows your plan and your connected devices, so it is a great opportunity for… for dependability, um, and cross-promoting other… other features and understanding our… our ecosystem. All in one place. That's it for me, and I think we're. It's Andrea next."
},
{
"time": "21:44:01",
"speaker": "Andrea Soto",
"text": "Okay, so I was thinking that we want to increase word of mouth, and stickers are a great way to do it. We also have our Easter egg. Um, logos, that would be great to… sell as stickers, so I added the option next to the logo when you click it to be able to buy the stickers on desktop and mobile. Or $2, so… That's my hack days project. Valent, you're next."
},
{
"time": "21:44:35",
"speaker": "Balint Farkas",
"text": "Hey, folks. So I'm working on the DuckDuckGo Chromium browser, also known as Duckium. We use Agentic workflows all the time, and it can, of course, see the code, but it cannot see the internals of the browser. So it can see the code, it can drive the browser via Chromium DevTools protocol, but it cannot see our own services like Sync. pixels, burn, feature flags, and so on. So I've exposed them to AI. You can just connect to the browser from your own AI terminal and then ask stuff, for example, why can't I see sync and backup in the menu? It should be here. The answer is because the feature flag in this live session isn't on. Or you can ask spicier stuff, such as performance search for I love cats, which it can do via Chromium DevTools protocol, and observe what pixels have been fired, which comes from the internals observation that I just added. And you can correlate these things and observe the browser's behavior as it happens, which will enable us more sophisticated debugging scenarios, and what's more important, more sophisticated Agentic workflows. That's my first thing. And the second thing is for Mark Sabu, who is out currently, and I'm sharing his video in a second."
},
{
"time": "21:45:52",
"speaker": "Andrea Soto",
"text": "Thank you."
},
{
"time": "21:46:02",
"speaker": "Balint Farkas",
"text": "Should be running. Hey folks! I'll walk you through the life of a Windows Release DRI in the next two minutes. After the last quick one days, we iterated on the design together with the release team, and I spent the last two days building it. The Windows Release Command Center. Until now, all of this was manual. We kept Asana and GitHub Actions in sync."
},
{
"time": "21:46:10",
"speaker": "Andrea Soto",
"text": "Ooh."
},
{
"time": "21:46:22",
"speaker": "Balint Farkas",
"text": "and dispatch every workflow by hand. Now the ODRI can check the state at a glance, and the release DRI can get through their duties by following the steps in the app. This is the state of every channel, which build is where, and at what rollout percentage. It all comes live from Asana. No more opening 5 tasks and 3 workflow pages to figure out where we are. The calendar shows what's coming up, branch cuts, rollout steps, so nobody gets surprised by a deadline. Clicking the release opens the DRI view. It shows the next section and flags anything that needs attention. Say we want to take 177 to 100% stable. I click this button and it dispatches the workflow. Because it's at 100%, the app asked me to follow these steps and upload it to the Microsoft Store. But I'm actually the DRI of 176, and I will need to do a hotfix. Hotfixing used to be the most manual part. Now I just pick from develop. Like PES, a PR that needs to be hotfixed. and just displays the bot with creation, which will actually just do this. I already done that to not block the demo. Umm. So I will need to upload the release notes. It loads the release notes automatically. So I can just approve it and upload. And now that it's done, I can just go back and simply follow the remaining steps to finish the release. And obviously, every flow syncs back into Asana, so the runbook still works if the app is down. That was Release Command Center. I hope it makes release weeks a little calmer and gives every DRI back some time for actual work. Oh, and sorry about the editing. I'm a dev, so this video was shipped as an MVP. Thanks, Odo! That's it for me."
},
{
"time": "21:48:10",
"speaker": "Alex Martemyanov",
"text": "Yeah, hey. Hey, everyone. This is me, Alex. There are so many new faces. I'm pretty sure some of you have no idea who I am. Which is fine, sometimes I'm not entirely sure myself. I'm Alex, from the Apple team, working on our macOS browser. And I thought I'd make it greater again. What would you expect from a macOS browser? During this hack… in the hack days. They say Julius was a hacker, performing live demos, chewing water into wine, accepting some cookies. Not bad for a hack days project, right? All right. Showtime. Could you please stop sharing?"
},
{
"time": "21:48:55",
"speaker": "Mark Ingram",
"text": "Thank you, Alan."
},
{
"time": "21:48:56",
"speaker": "Alex Martemyanov",
"text": "Got it."
},
{
"time": "21:48:57",
"speaker": "Balint Farkas",
"text": "Yeah, yeah, yeah, I'm trying, uh…"
},
{
"time": "21:48:59",
"speaker": "Julia Abreu",
"text": "It's a very…"
},
{
"time": "21:49:03",
"speaker": "Alex Martemyanov",
"text": "Yeah. Okay, first try failed miserably. Let's try again. Yeah, Kira. Hello. I hope you can hear the sound. Yeah, hope you can hear the sound. But seriously. What's happening to the world? AI is everywhere, too much information, too much liquid glass and beige. Sometimes I just want to burn it all down. Like this. Yeah, much better. Let's do it again. Where shall we start now? Sometimes I just want to run away back to something familiar. Or maybe look for some happiness. Or maybe I just need to talk to somebody. Yeah, maybe the problem isn't the internet. Maybe it's just how we see it. I wish I could change the whole settings of my life. how it appears. What if we could go back? Somewhere like this. And buttons look like buttons. Windows look like windows. Smells like Christmas morning, hot chocolate and mandarins, back when everybody was still around. Remember, on the websites. had some personality. When every page told you someone had paid attention to it. And speaking of the little things… You see? Even the tooltip. Respect your personal space. Your privacy. You just need to start and ask for some help. Here we are. Your protection, your privacy, your peace of mind. Thank you. That's all from me."
},
{
"time": "21:50:57",
"speaker": "Deepak",
"text": "I'll go next. Let's add the photo. Cool. Clearing my screen… What? I didn't work. Trying again. Okay. Can you all see my screen now?"
},
{
"time": "21:51:30",
"speaker": "John Boileau",
"text": "Yep."
},
{
"time": "21:51:31",
"speaker": "Julia Abreu",
"text": "We can see it."
},
{
"time": "21:51:32",
"speaker": "Deepak",
"text": "Okay. Cool. Playing a video, if you can't hear the audio, please. DuckyEye can generate a lot of great images, but only if you know exactly what to promote. ChatGPT, Gemini, and other tools have solved this with pre-existing templates for images, which has also led to a lot of virality for them. Well, now we have them too. If you go to new images, we have 15 of these different templates ranging from anime to 80s portraits to caricatures. Clicking on any of them populates the input field with the corresponding prompt. Some of them require you to upload a reference image, which is indicated by this icon, and then a user is notified to do that with this tooltip. I already did this one. So if you look at this, this was the reference image and I think it did a pretty good job. We also prompt the user to try more styles, so they can click on any of the, uh, styles, and then the original image gets, uh, pre-filled here with the. Uh, a defense bill. Another thing that really irked me was that after all of these images were generated, they were scattered across charts, and there was no one place to see them. Well, now we have this My Images tab. where you can see all of the different images that we have generated. You can delete one or all of them from here, and you can also go to the chart, where. they were generated, or use them as a reference for your next image. One cool thing that we do here is, if you do delete this image, then in the original conversation, where this was generated, we have this image deleted placeholder here, so that the continuity of the chat is not lost. That's it. Thank you. Yeah, that's it. Thank you."
},
{
"time": "21:53:20",
"speaker": "DR",
"text": "All right, so I go next. Let me see. Here. Can you see the screen? All right. So, uh, starting next Monday, our internal release, actually internal users, uh, in the internal release, will have, uh, the possibility of installing from the web, uh, Chrome Web Store 1Password. And this video showcases how it works, that it connects to the desktop app that you have installed. You can see that it works for registering your email and autofilling your password for it. You have to unlock it first, though. You can see here how it works with Touch ID. Um… You can fill the login, password. Then it's going to ask you to save it. You can also, of course, use it to log in to a site, those details. So you guys see here. Then it also works for passkeys, where you can register a passkey for one of the accounts that you already have in 1Password. Um… Again, it's going to probably ask for authentication. And then use that same passkey. To log into the site. In this example, it didn't ask for additional authentication because it was so close to the previous one, but it's working quite nicely. And we'll be posting an announcement with details on how you can provide feedback for this. I said it's quite complicated on the technical side. Thank you. Hold on."
},
{
"time": "21:54:58",
"speaker": "AO",
"text": "Hey, Elle. This is a demo that Alex, Elle, and I did. So double Alex demo. or DACAI artifacts. Hey everyone! During Hack Days, we explored DACAI artifacts. Documents, presentations, and visuals. But you know what? It's boring, so here's the twist. This presentation was built entirely in DACAI. See? It's right here, inside the chat. And you know what? I can even download it as a PowerPoint file. Now check this out. Let's ask how much I could save. I get an answer, but only for this one case. Different numbers, we need to try again. Or I can get an app that does that for me. Duck AI builds it right here, next to the chat. You can watch it come together. Not a code block, not a screenshot, a real working app. Now I can change the amount, the time, the rate, and it updates instantly. Have a PDF? You can turn it into just about anything. Study notes, a brief, key takeaways, whatever you need, as a document. You have full control over your doc and markdown. You can also let Duck AI rework it. When you are done editing, you can now export your doc. With Artifacts, you can build your own website too. If you want something different, you can just point DuckAI at it and ask for an edit. Every version is saved, so if you don't like your change, you can go back in one click. Artifacts also work well with our browser features. Let's say I have a really long article, and absolutely no desire to read all of it. Attach the top, and Dakia will summarize the main takeaways visually. Just a few seconds, and it's ready, clear, structured, and much easier to understand. And of course, artifacts work great on mobile too. Try them yourself! That's it. Thank you."
},
{
"time": "21:57:18",
"speaker": "Gary K",
"text": "All right, I'm up. Let me share with sound. a video… Here we go. Any time there is an incident, the first question is always, what changed? And the answer is potentially in a lot of different places. So I've created a command line tool that will aggregate the events that are likely to be interesting for anyone investigating new behavior. What changed is a tool that can access Azure activity logs, Asana deployment plans, Jenkins jobs, GitHub pull requests and actions, REX commands, and Mattermost notifications. Because there can be a lot of data in these sources, the tool ships with default settings that are designed to give high signal-to-noise ratio. These settings can be overwritten via a config file. or a runtime flag. Users can search backwards from now or over a specific window of time, from minutes to days. There are a host of flags designed to make it easy to use. Users can opt in to format the output as JSON and perform further filtering. and transformations using JQ. What changed uses your own credentials, so security posture is maintained. What Changed is available now at www.duckduckgo.com/gcurtis/whatchanged. Thank you. Like and subscribe for more great content. Thank you."
},
{
"time": "21:58:52",
"speaker": "Tom Strba",
"text": "Nice. Hey everyone. So over the last couple of weeks in OJ and OM, we have been trying to build tools for Duck AI. What does it mean? We would like Duck AI to be able to get context automatically from your browser. And not only that, we would like to. Duck AI to be able to control the browser. So, during hack days, I tried to put all the pieces together, and uh… I would like to show you a proof of concept that I managed to build, so… Let's say I have, uh… Couple of, uh… shoes I am trying to buy. And I would like Duck AI to recommend me which shoes should I buy. So… I will ask you. Which shoes should I buy from all these options? And I'm not providing more context. It will basically get the list of my tabs. It will read the content and it will recommend me, oh, this is the best for speed for daily trainer. whatever but this is not where the fun ends this is where the fun starts so how about now we ask Duck AI. to buy our choice. So for example, buy me stride arrow three. And Duck AI will basically open a new tab and start buying shoes for me. As you can see, here is the store and it's starting to fill the form and it doesn't know which. shoe size it should buy, so it asks me. I will just respond, for example. 10, and it now reads the form. As you can see, it filled. The 10, and now there is a… the website needs the address. So the fun part is that we already have the identity in the browser. So Dakia asks me, hey, can I use this identity from the browser to fill it? And yes, of course, let's share it. If we take a look at the store… by the way, this is the store I built for myself for testing. The address is now filled. and it asks me, it asks me last time, hey, how about, uh, I'm gonna, I'm gonna submit the form, are you, are you sure? Yes. I am sure. And my shoes are now ordered. I didn't need to click on that website. I just wanted you to see what is happening in the background. But Duck AI now or soon will be able to buy. stuff for you or do anything in the browser you want it to do. Uh, that's the first demo I wanted to show you. And the second one, this one will be really quick, and it's, like, internal tool enhancement. Uh, if you… Often test a Mac. make ship reviews, you know the process. You need to download the DMG, attach it manually. Then drag and drop it. The instance is running, so you need to quit it. Then drag and drop again, wait for installation and then run it. So I build the button and now you will just click to run and you can start reading the. ship review, and everything will happen in the background, and after the build is downloaded, I'm gonna speed this up a little bit, it will basically just runs, and you can start testing. Um, that's it for me, thank you."
},
{
"time": "22:02:22",
"speaker": "I(",
"text": "All right. I'm just getting my screen up. Please let me know if you can see it. All right, looks like it's there. All right. So as a user, why should I care about the protections report on our browser? It's a list of numbers and icons. What is the actual benefit? So I reworked it. I started with the bottom line. Companies are watching you. It's creepy. Also highlighted right away that there's an issue to be fixed. Privacy isn't your default, and here's why that matters. The more of your everyday browsing happens in our app, the less these companies get to see. So let's expand this. Immediately, we get a creepy, interactive, prying eye that follows your cursor around. And we get useful info. We block tracking attempts, but why? They target you with ads, they watch what you read, and they record what you click. It's creepy. So who's watching? Well, Google does a lot of spying, and it was on 4 out of the 5 sites he visited. And Facebook was on almost half. Again. It's really creepy. So I wondered how we could make this more like a personal infographic. Here we highlight the nosiest site of the week, and we show a ring of trackers from that site that blink like eyes to drive home the visceral idea that you're being watched. Now, we empower you to understand some of what we're doing by showing you how we prevent a digital fingerprint view from being created. You can hover over each component of the fingerprint and see what we've hid from creepy companies. And how many times? And finally, we contrast this with Chrome. Where companies swarm around your cursor, and cookie pops and ads fill the screen. We drive home the fact that Chrome is owned by Google, and that unlike on Chrome, DuckDuckGo blocks Google's tracking attempts. So maybe this really creeped you out. Why are so many companies watching you read the news? Let's share that. And just like that, a promotional infographic can be shared with a link to download our browser. So try it out for yourself on Windows on the link task. Thanks."
},
{
"time": "22:04:41",
"speaker": "Vinay Nadig",
"text": "All right, that's me next. Let me share my screen. Alright, so for hack days, I have. built an app called, QuackTime. What it does is, it lets you schedule ad hoc meetings, through Mattermost without having to do the whole song and dance of matching time zones across. uh, different geographies. So, to use that, go to QuackTime, go to Metamost, or any DM, QuackTime and DM duration, in this case 60 minutes, and then we'll set a type build. You will get a link, you can open it. you will end up on this interface, basically. You will see that all your, uh… uh, working hours and free, uh, slots through Outlook has already been filled in. And that would be the same case for another person as well. So, the other person will also open the same link. And yeah, so you can see that I'm working with Adrian, so I have this much overlap with him, so you can select whichever task that you want. And, oh, sorry. You can propose it. And the other person will… See an option to accept that. And once you accept it, an invite will be sent automatically at the correct time, along with the Zoom link for… which uses your personal Zoom link. That's pretty much it. I'll stop there."
},
{
"time": "22:06:17",
"speaker": "Nicklas Jensen",
"text": "All right, I'm next up here, and I am going to be sharing some porn. out of a conversation, actually, in a Zoom chat. So, one of the things on the meetup that happened was that we, uh… At least for a lot of the ducklings, we were freaking out about how much merch and how many things have been over the years. So the idea was to create a hall of fame for specifically T-shirts. So Miguel and I worked on that. For this, uh… For these hack days, the goal is you have a timeline, you can see all of the different designs that have been over the years. These are all the ones that Miguel could get a hold of so far. You can vote for your favorite ones. And also, if you don't feel like going through them all manually, you can just… pick out of sort of a random generated thing. And yeah, that's the T-shirt Hall of Fame. How do I stop sharing? Thank you."
},
{
"time": "22:07:30",
"speaker": "Dominik Kapusta",
"text": "All right, here's a demo of the… Long overdue feature on… macOS, which is the reader mode. Um… Reader mode. extracts text from websites. And here you can see that, uh, we're navigating to a website that, uh, is eligible for being displayed in reader mode, which contains relatively more text, uh, and, um… Just with a single button in the address bar, um… we turn on the reader mode, then we navigate away from that website, it just resets to a regular view. here, uh, you can see, uh, visiting another website, uh, that's also eligible for reader mode, and here's, uh, here's how it works, but it's not all, because I've worked on that, uh, during Hug Days, but. Um… a parallel to my work, actually. Marios was doing the same thing for iOS, so here it is, um… Here, the reader mode, uh… Button is in the website, uh… menu… And this demo will also show you that it respects the browser theme. So that, um… shows correctly in the dark theme as well. Even if you visit a website that… uh, doesn't honor the system theme, uh, then the reader mode, of course, uh, will be dark in this case, when the Guardian was in, uh, in light theme. Yeah, and this all was possible. to do thanks to a library by Meek that extracts. text from, from websites. So really the integration. It was fairly minimal here, and it was just a bit of work. That's why I said it was long overdue, because that library was published a few months ago already. That's it. Thank you."
},
{
"time": "22:10:01",
"speaker": "Valerie Kraucunas",
"text": "Alrighty, hello. Um, so, I'm Valerie. I work on the front-end team. Um, the thing I've been really heavily focused on all year is the rebrand, and that… with that has come… turning over, uh, style values from one… one way to a new way. Um… with that, we have these things called design tokens, which is basically, like, a codified design decision. And so, when we're styling… the new rebranded stuff we're using, or at least we want to be using all of these things. That said, um, a lot of times when you are handed designs that are rebranded. You end up with this scenario of… It'll be a mix of using the new style tokens and the old style tokens. So in this case, like the SDS styles are a legacy thing. Um, and I want to be very clear, this is not a shame. This is not a shame campaign or anything on designers, because oftentimes when they are building these things, they are… they have the old way in the same file as the. new way. And so it's really, really tedious to get down into every single nook and cranny and node or layer to make sure that you're using the new styles on everything and to make it even more complicated. So when developers see this, we just see this and we're like, why is this still here? Why are we still using SDS things? But when you get over into the design look, so this first column is when they're picking the first padding value, so space X07. When they're picking the second column value, space X03. They look the same. It's really easy to co-mingle the old style and the new. And so, um, basically my concept is to… be able to hand a Claude skill a Figma link, and have it go and traverse the entire. All child nodes of that, um… of whatever you linked over. So. I'm going to try this. I doubt it will… I doubt it'll work. It does not go as fast as I would like it to, but we'll give it a go. So, basically, what happens is you, you know, invoke it with the Figma link, then you give it an Asana link. And give it a name. Um… What that ends up spitting out is a nice report for designers to be able to go and update all the things that are using, uh, old values, or possibly things that are just out of system. As well, um, now I want to be clear, I'm standing on the shoulders of giants here, because, um, while we've been doing the SERP rebrand, uh, my partner in crime, Rachel Lim, created a… skill called Marlin Rebrando, which is basically a dictionary of, uh, knowledge for this was the old way, this is the new way, these are the things we should be checking for when we are executing the rebrand and the code. Um, so my skill is heavily relying on that, but, um… What it'll basically do is it will just… fully traverse like a node tree, if you will, all the way down. Um… And yeah, the things that we're looking for are the use of legacy variables. This is run from within the DDG. repo, for now, at least. So what it will do also is it'll look at the currently installed design token package and make sure that there aren't values being used in the designs that don't exist in. the installed package, then also looking for just colors that aren't tied to a system value, unselled text, et cetera. So what this ends up doing is it will create a. design token report, and. give the designers a link to the layer that. his, uh… has a property on it that's a problem, if you will, and then give it an idea of, like, this is an unexpected value, and then has an expected or a suggested value. The nice part about this as well is that developers. don't necessarily have to wait on designers to go. What?"
},
{
"time": "22:14:28",
"speaker": "Julia Abreu",
"text": "Well… I'm so sorry, I hate to do it, but I have to interrupt you, because we're at 4 minutes now."
},
{
"time": "22:14:35",
"speaker": "Valerie Kraucunas",
"text": "Oh, shoot, I'm so sorry. Okay, well…"
},
{
"time": "22:14:36",
"speaker": "Julia Abreu",
"text": "Sorry."
},
{
"time": "22:14:38",
"speaker": "Valerie Kraucunas",
"text": "Making everybody's life better, that's all."
},
{
"time": "22:14:50",
"speaker": "Stano",
"text": "Okay, I believe I was next. Um, I'm gonna… I hope you can see my screen now. Someone please confirm."
},
{
"time": "22:15:00",
"speaker": "Julia Abreu",
"text": "No, we cannot."
},
{
"time": "22:15:01",
"speaker": "Jason Ashby",
"text": "Nope."
},
{
"time": "22:15:02",
"speaker": "Stano",
"text": "You cannot? Let me try again. Where is… The sun got stuck."
},
{
"time": "22:15:24",
"speaker": "Julia Abreu",
"text": "Did you find the share button?"
},
{
"time": "22:15:27",
"speaker": "Stano",
"text": "Give me a sec, I reconnected. You can take it out of my budget, I'll make it quick. It does not want to work. You can skip me. Go on."
},
{
"time": "22:15:45",
"speaker": "Anh Do",
"text": "Okay, looks like I'm next. So let me try to share my screen. Wow. Uh, how do I share, like, a… Particular windows, um… Sorry, um, I had so many windows opening at the moment. Oh, come on. Guys, it is not fun. Sorry. OK, I'm going to share my desktop then. One sec. Hopefully, you can see that on my screen. Can anyone see it? Okay. So for context, in OK, we are helping people in the US to remove personal information from data broker by submitting our request on their."
},
{
"time": "22:16:21",
"speaker": "Julia Abreu",
"text": "I can see it."
},
{
"time": "22:16:33",
"speaker": "Anh Do",
"text": "behalf. But recently, we have a lot of issues with broker block our archive with CloudFlare and CAPTCHA, so we can submit the form for the users. So this demo has been built based on Ben Smith design prototype. Let me play that."
}
]
}
@@ -0,0 +1,63 @@
---
title: "All-Company Meeting"
date: null
zoom_note_id: "bPprGg5DTye9H-RQXkwMuw"
source: "https://us01mynotes.zoom.us/bPprGg5DTye9H-RQXkwMuw"
---
# All-Company Meeting
[Zoom My Notes](https://us01mynotes.zoom.us/bPprGg5DTye9H-RQXkwMuw)
## Transcript
- **21:29:37 · John Boileau:** And this way.
- **21:30:59 · Gabriel Weinberg:** Hello, everybody. We have a few quick updates that we're going to mainly do the Hack Day's showcase today that Julia's leading. Everyday AI Guild. That's Zac. Okay. Yeah, there it is. All right. Yeah.
- **21:31:17 · Zach:** Yep. Uh, yeah, sorry. Um, yeah, so just quick announcement there. Uh, Julio's moving to the, uh, DRI for that, um, and Mario is going to be, uh, the secondary. Um, so they've both been pretty heavily involved in, uh, that process, and we're excited for this transition.
- **21:31:40 · Josh Frank:** I believe I'm next. So hey, everybody. I'm taking a quick break from the gym where you can frequently find me to tell you about random access learning. Just put the link in the chat. We have a session Thursday next week. 12:00 PM Eastern time, I think that's US Eastern, on corporate sustainability and being carbon negative in the AI era. So please, everybody, come to that if you can.
- **21:32:17 · AC:** Everybody Ă last week we had our away mobile browser experience. Assessment our goals days a same they car our Browser mobile, browser experience from good to great in the previous cycle. We ship the android. And dialog.
- **21:32:55 · AC:** Internal the keyboard now always on the new tape. Device to device migration. We want to ship this before black friday. A ui that s case for split. Screen. And we want to what support for many bar. We are going to start also the new tupage experiment for new users. And they think next these mark.
- **21:34:04 · Mark Ingram:** Hey, everyone. I'm Mark from OL, the Desktop Browsers Objective. Some exciting news this week is Ivy is taking over as the ODRI, and I'll be moving on to the advisor role. I'll keep it short because I know we're on hack days this week, but I also, just before introing Ivy, I want to announce my new midlife crisis to everybody, which is. I decided to start learning the drums, and I can see Carl's excited face on the screen. Okay, that's my midlife crisis update. Over. Ivy, do you want to say hi?
- **21:34:31 · I(:** Yeah, I'm just really excited to take over OL, and stay tuned for some exciting news about extensions on macOS coming really soon.
- **21:34:43 · Julia Abreu:** All right. Congrats, Ivy. That's exciting. So I'm kicking us off today. I'm going to actually share my whole screen because I'm going to be moving from one side to the other. So I have two demos today. So we have the Julia special, two demos. Um, my first demo is… since the meetup, I've been thinking about how, um… We need to, like, I've been thinking about the new objectives that we have about word of mouth and how we talk about DuckDuckGo and how, especially for ducklings, is hard to answer some of the questions that we get once you first join DuckDuckGo. So I wanted to create something that helps them answer those questions and understand our products better. So I do want to shout out Ben Smith because he created this. I just basically said the idea. Um, so this is an artifact, um, and you will see the different tabs here, so you have different questions per tab. Um, and then you have the specific questions here, which you can click on and see them in writing. Or, if you want to, you can actually listen to it as. kind of like a podcast episode. Um, so… let me know if you can actually hear this. I'm just gonna play a couple of seconds. Episode 1. The Big Picture. Last week, a friend of mine looked up, is it normal for your knee to click? Just once. For the next week, every website they visited tried to sell them knee supports, walking sticks, and one of those… So you get the gist of it. I hope you got to hear that. Um, so this is still a work in progress. We're going to be creating this content. Um, I really didn't want to… to be in audio, because I feel like if you get that in writing, it's different explaining as a human. So, this is what we're building. And then, the second thing I'm building… dang, the Zoom… The Zoom thing doesn't let me go to the other one. Okay, here. So this is the second thing that I built. This is coming from feedback from people who went on leave, which was around how to catch up with everything that had been happening since they were gone. Um, so this is a how… what do you miss? Uh, so basically, you add here the date you started your leave, and then the start of the date you return. Um, you click catch me up. Uh, it takes a couple of seconds, but it basically gives you, like, a summary of a bunch of things that happened while you were out. So, um… company-wide posts, uh, what was published, what was answered in odds and ends, so all the Q&A, which objectives are up-to-date, like, which assessment was the last thing, and what we're working on right now, uh, things that were shipped, uh, also, like, new AORs, retired AORs, so here, as you can see, like, one, um, sample of, like, the announcements. It gives you just, like, a summary of all the announcements. Um, and we also have a people tab, so, like, new hires, which teams they are in, uh, people who left, and then it also shows you, um, green bow ties, spotlights, in case you want to catch up with all the spotlights that were posted that. Um, the time you were away. I do want to add a tab for specifically functional team updates, so, like, if there's a new process or anything specific to the functional team, I'm just not sure yet where to get that information from. Um, so… working on that. If you have suggestions, or if you would like to be… try out these things, let me know. I would love the feedback. Um, and now, this is it for my demos. Now I have a demo by, um, Robert, who couldn't be here. So, also, please ping me if you can't, uh, hear this. Hello, hello, hello. I worked on three hacks this time around. I'll try to get through it all quickly. The first is to do with increasing Duck AI prompts. I thought that maybe a nice way to do that would be to add a bit of discoverability into our browser. So if you are looking at a foreign language page, then you get a suggestion to translate it. Can someone, uh, unmute themselves and just let me know if you can hear this? Because I'm, like, concerned because I can't see that. Okay, thank you.
- **21:38:59 · Anne Hildebrand:** Yes.
- **21:38:59 · Zaahir Moolla:** We can hear.
- **21:39:02 · Julia Abreu:** And, um, so yep, that sends the page content off into Duck AI and translates it. We already had this feature, but, you know, just adding the prompt up there. That works. And there's lots of… this is very exploratory. There's lots of different ideas along these lines that we could look into, such as, if it's a long article, maybe we, um, tell the user how long it'll take and prompt to summarize, or on a shopping page, we could prompt them to, um, compare. Uh, the cool thing is that these suggestions here, these three options, that's all generated by our local LLM, so it's completely private. Um, nothing's, uh, hitting our servers until you click on one of those. Um, most recipes too. And this is a cool one, if you, um, are looking at an event, you can ask DuckAI to compute the event details. And I added this, um… dynamic UI, this card at the bottom of the Doug AI answer, and it gives you a button to add it to your calendar. Uh, one cool idea is, if you're looking at a privacy page, it could offer to help the user understand what it is they're agreeing to, so very within our brand. And another cool idea, just because YouTube features always seem to perform really well for us, is we could, um, prompt the user to, uh, do things using the transcript that appears in YouTube here. So this actually extracts the transcript from this panel. And it can send it to DocAI. Okay. Hack number two is, um, fixing a personal frustration of mine. So, I'm a Raycast user, and I'm always doing things like… um, converting currencies and, like, complex queries like this. And, uh, it frustrates me to no end that, um, that doesn't work when you plug it into DuckDuckGo. And so what I did is I had Claude reverse engineer the raycast calculator, and I've plopped that into SERP, and so now you can, um, jump into SERP running on this test machine, and do complex queries like that. And there's all sorts of things you can do. Mixed currencies is kind of cool. Um, one thing I use a lot is, like. Adding different units. And there's also, like, sort of… Complex math bits that are kind of fun. And the way that I handled this in the UI is that —. The simple calculator stays if you're doing a simple expression. And same if you're just doing a regular. conversion, then we get the existing UI for that too, because it's nicer since it has the dropdowns. Okay, and my third hack is yet another frustration that I've had for a while. So here we are looking at my phone. Oops. And, um, so we've always been able to switch tabs by swiping on the address bar at the bottom there, and that is quite nice. But what we haven't been able to do is swipe up to go to the tab switcher. And so I added that, and it turns out that Bartosz had already been working on this, so he has helped me a lot with this. Thank you very much. And yeah, it's really nice that you can… it tracks your finger, so you can kind of drag out halfway. Real smooth. And Bartosz added, you can drag out halfway and swipe to the right. Those are my hacks. Thank you very much. All right, and we're not done yet. I have another one by Pablo. When people want their data on another device they don't go looking for things and sync it back up doesn't really tell them what it does. Chanel's usability insight report made that very clear. And the problem is getting bigger. Zac AI chats, uh, can easily be lost. Um, the more stuff we add to Zac AI, um, the more. things that are worth keeping, Sink and Backup will just not be enough. So, we are proposing MyTacticGo, a private account. It's private because it's from us, and it needs no personal data to sign in, so no email and no password to remember. It is encrypted, your data is backed up, unsynced, and we show what's ready to go, like 100 passwords or 244 bookmarks, so the value is very concrete. On desktop, MyTacticGo lives in the toolbar, where people actually look. It lists what will be synced, like your chats and passwords and bookmarks, but it also brings your subscription. So, this can become, actually, your home base, and it shows your plan and your connected devices, so it is a great opportunity for… for dependability, um, and cross-promoting other… other features and understanding our… our ecosystem. All in one place. That's it for me, and I think we're. It's Andrea next.
- **21:44:01 · Andrea Soto:** Okay, so I was thinking that we want to increase word of mouth, and stickers are a great way to do it. We also have our Easter egg. Um, logos, that would be great to… sell as stickers, so I added the option next to the logo when you click it to be able to buy the stickers on desktop and mobile. Or $2, so… That's my hack days project. Valent, you're next.
- **21:44:35 · Balint Farkas:** Hey, folks. So I'm working on the DuckDuckGo Chromium browser, also known as Duckium. We use Agentic workflows all the time, and it can, of course, see the code, but it cannot see the internals of the browser. So it can see the code, it can drive the browser via Chromium DevTools protocol, but it cannot see our own services like Sync. pixels, burn, feature flags, and so on. So I've exposed them to AI. You can just connect to the browser from your own AI terminal and then ask stuff, for example, why can't I see sync and backup in the menu? It should be here. The answer is because the feature flag in this live session isn't on. Or you can ask spicier stuff, such as performance search for I love cats, which it can do via Chromium DevTools protocol, and observe what pixels have been fired, which comes from the internals observation that I just added. And you can correlate these things and observe the browser's behavior as it happens, which will enable us more sophisticated debugging scenarios, and what's more important, more sophisticated Agentic workflows. That's my first thing. And the second thing is for Mark Sabu, who is out currently, and I'm sharing his video in a second.
- **21:45:52 · Andrea Soto:** Thank you.
- **21:46:02 · Balint Farkas:** Should be running. Hey folks! I'll walk you through the life of a Windows Release DRI in the next two minutes. After the last quick one days, we iterated on the design together with the release team, and I spent the last two days building it. The Windows Release Command Center. Until now, all of this was manual. We kept Asana and GitHub Actions in sync.
- **21:46:10 · Andrea Soto:** Ooh.
- **21:46:22 · Balint Farkas:** and dispatch every workflow by hand. Now the ODRI can check the state at a glance, and the release DRI can get through their duties by following the steps in the app. This is the state of every channel, which build is where, and at what rollout percentage. It all comes live from Asana. No more opening 5 tasks and 3 workflow pages to figure out where we are. The calendar shows what's coming up, branch cuts, rollout steps, so nobody gets surprised by a deadline. Clicking the release opens the DRI view. It shows the next section and flags anything that needs attention. Say we want to take 177 to 100% stable. I click this button and it dispatches the workflow. Because it's at 100%, the app asked me to follow these steps and upload it to the Microsoft Store. But I'm actually the DRI of 176, and I will need to do a hotfix. Hotfixing used to be the most manual part. Now I just pick from develop. Like PES, a PR that needs to be hotfixed. and just displays the bot with creation, which will actually just do this. I already done that to not block the demo. Umm. So I will need to upload the release notes. It loads the release notes automatically. So I can just approve it and upload. And now that it's done, I can just go back and simply follow the remaining steps to finish the release. And obviously, every flow syncs back into Asana, so the runbook still works if the app is down. That was Release Command Center. I hope it makes release weeks a little calmer and gives every DRI back some time for actual work. Oh, and sorry about the editing. I'm a dev, so this video was shipped as an MVP. Thanks, Odo! That's it for me.
- **21:48:10 · Alex Martemyanov:** Yeah, hey. Hey, everyone. This is me, Alex. There are so many new faces. I'm pretty sure some of you have no idea who I am. Which is fine, sometimes I'm not entirely sure myself. I'm Alex, from the Apple team, working on our macOS browser. And I thought I'd make it greater again. What would you expect from a macOS browser? During this hack… in the hack days. They say Julius was a hacker, performing live demos, chewing water into wine, accepting some cookies. Not bad for a hack days project, right? All right. Showtime. Could you please stop sharing?
- **21:48:55 · Mark Ingram:** Thank you, Alan.
- **21:48:56 · Alex Martemyanov:** Got it.
- **21:48:57 · Balint Farkas:** Yeah, yeah, yeah, I'm trying, uh…
- **21:48:59 · Julia Abreu:** It's a very…
- **21:49:03 · Alex Martemyanov:** Yeah. Okay, first try failed miserably. Let's try again. Yeah, Kira. Hello. I hope you can hear the sound. Yeah, hope you can hear the sound. But seriously. What's happening to the world? AI is everywhere, too much information, too much liquid glass and beige. Sometimes I just want to burn it all down. Like this. Yeah, much better. Let's do it again. Where shall we start now? Sometimes I just want to run away back to something familiar. Or maybe look for some happiness. Or maybe I just need to talk to somebody. Yeah, maybe the problem isn't the internet. Maybe it's just how we see it. I wish I could change the whole settings of my life. how it appears. What if we could go back? Somewhere like this. And buttons look like buttons. Windows look like windows. Smells like Christmas morning, hot chocolate and mandarins, back when everybody was still around. Remember, on the websites. had some personality. When every page told you someone had paid attention to it. And speaking of the little things… You see? Even the tooltip. Respect your personal space. Your privacy. You just need to start and ask for some help. Here we are. Your protection, your privacy, your peace of mind. Thank you. That's all from me.
- **21:50:57 · Deepak:** I'll go next. Let's add the photo. Cool. Clearing my screen… What? I didn't work. Trying again. Okay. Can you all see my screen now?
- **21:51:30 · John Boileau:** Yep.
- **21:51:31 · Julia Abreu:** We can see it.
- **21:51:32 · Deepak:** Okay. Cool. Playing a video, if you can't hear the audio, please. DuckyEye can generate a lot of great images, but only if you know exactly what to promote. ChatGPT, Gemini, and other tools have solved this with pre-existing templates for images, which has also led to a lot of virality for them. Well, now we have them too. If you go to new images, we have 15 of these different templates ranging from anime to 80s portraits to caricatures. Clicking on any of them populates the input field with the corresponding prompt. Some of them require you to upload a reference image, which is indicated by this icon, and then a user is notified to do that with this tooltip. I already did this one. So if you look at this, this was the reference image and I think it did a pretty good job. We also prompt the user to try more styles, so they can click on any of the, uh, styles, and then the original image gets, uh, pre-filled here with the. Uh, a defense bill. Another thing that really irked me was that after all of these images were generated, they were scattered across charts, and there was no one place to see them. Well, now we have this My Images tab. where you can see all of the different images that we have generated. You can delete one or all of them from here, and you can also go to the chart, where. they were generated, or use them as a reference for your next image. One cool thing that we do here is, if you do delete this image, then in the original conversation, where this was generated, we have this image deleted placeholder here, so that the continuity of the chat is not lost. That's it. Thank you. Yeah, that's it. Thank you.
- **21:53:20 · DR:** All right, so I go next. Let me see. Here. Can you see the screen? All right. So, uh, starting next Monday, our internal release, actually internal users, uh, in the internal release, will have, uh, the possibility of installing from the web, uh, Chrome Web Store 1Password. And this video showcases how it works, that it connects to the desktop app that you have installed. You can see that it works for registering your email and autofilling your password for it. You have to unlock it first, though. You can see here how it works with Touch ID. Um… You can fill the login, password. Then it's going to ask you to save it. You can also, of course, use it to log in to a site, those details. So you guys see here. Then it also works for passkeys, where you can register a passkey for one of the accounts that you already have in 1Password. Um… Again, it's going to probably ask for authentication. And then use that same passkey. To log into the site. In this example, it didn't ask for additional authentication because it was so close to the previous one, but it's working quite nicely. And we'll be posting an announcement with details on how you can provide feedback for this. I said it's quite complicated on the technical side. Thank you. Hold on.
- **21:54:58 · AO:** Hey, Elle. This is a demo that Alex, Elle, and I did. So double Alex demo. or DACAI artifacts. Hey everyone! During Hack Days, we explored DACAI artifacts. Documents, presentations, and visuals. But you know what? It's boring, so here's the twist. This presentation was built entirely in DACAI. See? It's right here, inside the chat. And you know what? I can even download it as a PowerPoint file. Now check this out. Let's ask how much I could save. I get an answer, but only for this one case. Different numbers, we need to try again. Or I can get an app that does that for me. Duck AI builds it right here, next to the chat. You can watch it come together. Not a code block, not a screenshot, a real working app. Now I can change the amount, the time, the rate, and it updates instantly. Have a PDF? You can turn it into just about anything. Study notes, a brief, key takeaways, whatever you need, as a document. You have full control over your doc and markdown. You can also let Duck AI rework it. When you are done editing, you can now export your doc. With Artifacts, you can build your own website too. If you want something different, you can just point DuckAI at it and ask for an edit. Every version is saved, so if you don't like your change, you can go back in one click. Artifacts also work well with our browser features. Let's say I have a really long article, and absolutely no desire to read all of it. Attach the top, and Dakia will summarize the main takeaways visually. Just a few seconds, and it's ready, clear, structured, and much easier to understand. And of course, artifacts work great on mobile too. Try them yourself! That's it. Thank you.
- **21:57:18 · Gary K:** All right, I'm up. Let me share with sound. a video… Here we go. Any time there is an incident, the first question is always, what changed? And the answer is potentially in a lot of different places. So I've created a command line tool that will aggregate the events that are likely to be interesting for anyone investigating new behavior. What changed is a tool that can access Azure activity logs, Asana deployment plans, Jenkins jobs, GitHub pull requests and actions, REX commands, and Mattermost notifications. Because there can be a lot of data in these sources, the tool ships with default settings that are designed to give high signal-to-noise ratio. These settings can be overwritten via a config file. or a runtime flag. Users can search backwards from now or over a specific window of time, from minutes to days. There are a host of flags designed to make it easy to use. Users can opt in to format the output as JSON and perform further filtering. and transformations using JQ. What changed uses your own credentials, so security posture is maintained. What Changed is available now at www.duckduckgo.com/gcurtis/whatchanged. Thank you. Like and subscribe for more great content. Thank you.
- **21:58:52 · Tom Strba:** Nice. Hey everyone. So over the last couple of weeks in OJ and OM, we have been trying to build tools for Duck AI. What does it mean? We would like Duck AI to be able to get context automatically from your browser. And not only that, we would like to. Duck AI to be able to control the browser. So, during hack days, I tried to put all the pieces together, and uh… I would like to show you a proof of concept that I managed to build, so… Let's say I have, uh… Couple of, uh… shoes I am trying to buy. And I would like Duck AI to recommend me which shoes should I buy. So… I will ask you. Which shoes should I buy from all these options? And I'm not providing more context. It will basically get the list of my tabs. It will read the content and it will recommend me, oh, this is the best for speed for daily trainer. whatever but this is not where the fun ends this is where the fun starts so how about now we ask Duck AI. to buy our choice. So for example, buy me stride arrow three. And Duck AI will basically open a new tab and start buying shoes for me. As you can see, here is the store and it's starting to fill the form and it doesn't know which. shoe size it should buy, so it asks me. I will just respond, for example. 10, and it now reads the form. As you can see, it filled. The 10, and now there is a… the website needs the address. So the fun part is that we already have the identity in the browser. So Dakia asks me, hey, can I use this identity from the browser to fill it? And yes, of course, let's share it. If we take a look at the store… by the way, this is the store I built for myself for testing. The address is now filled. and it asks me, it asks me last time, hey, how about, uh, I'm gonna, I'm gonna submit the form, are you, are you sure? Yes. I am sure. And my shoes are now ordered. I didn't need to click on that website. I just wanted you to see what is happening in the background. But Duck AI now or soon will be able to buy. stuff for you or do anything in the browser you want it to do. Uh, that's the first demo I wanted to show you. And the second one, this one will be really quick, and it's, like, internal tool enhancement. Uh, if you… Often test a Mac. make ship reviews, you know the process. You need to download the DMG, attach it manually. Then drag and drop it. The instance is running, so you need to quit it. Then drag and drop again, wait for installation and then run it. So I build the button and now you will just click to run and you can start reading the. ship review, and everything will happen in the background, and after the build is downloaded, I'm gonna speed this up a little bit, it will basically just runs, and you can start testing. Um, that's it for me, thank you.
- **22:02:22 · I(:** All right. I'm just getting my screen up. Please let me know if you can see it. All right, looks like it's there. All right. So as a user, why should I care about the protections report on our browser? It's a list of numbers and icons. What is the actual benefit? So I reworked it. I started with the bottom line. Companies are watching you. It's creepy. Also highlighted right away that there's an issue to be fixed. Privacy isn't your default, and here's why that matters. The more of your everyday browsing happens in our app, the less these companies get to see. So let's expand this. Immediately, we get a creepy, interactive, prying eye that follows your cursor around. And we get useful info. We block tracking attempts, but why? They target you with ads, they watch what you read, and they record what you click. It's creepy. So who's watching? Well, Google does a lot of spying, and it was on 4 out of the 5 sites he visited. And Facebook was on almost half. Again. It's really creepy. So I wondered how we could make this more like a personal infographic. Here we highlight the nosiest site of the week, and we show a ring of trackers from that site that blink like eyes to drive home the visceral idea that you're being watched. Now, we empower you to understand some of what we're doing by showing you how we prevent a digital fingerprint view from being created. You can hover over each component of the fingerprint and see what we've hid from creepy companies. And how many times? And finally, we contrast this with Chrome. Where companies swarm around your cursor, and cookie pops and ads fill the screen. We drive home the fact that Chrome is owned by Google, and that unlike on Chrome, DuckDuckGo blocks Google's tracking attempts. So maybe this really creeped you out. Why are so many companies watching you read the news? Let's share that. And just like that, a promotional infographic can be shared with a link to download our browser. So try it out for yourself on Windows on the link task. Thanks.
- **22:04:41 · Vinay Nadig:** All right, that's me next. Let me share my screen. Alright, so for hack days, I have. built an app called, QuackTime. What it does is, it lets you schedule ad hoc meetings, through Mattermost without having to do the whole song and dance of matching time zones across. uh, different geographies. So, to use that, go to QuackTime, go to Metamost, or any DM, QuackTime and DM duration, in this case 60 minutes, and then we'll set a type build. You will get a link, you can open it. you will end up on this interface, basically. You will see that all your, uh… uh, working hours and free, uh, slots through Outlook has already been filled in. And that would be the same case for another person as well. So, the other person will also open the same link. And yeah, so you can see that I'm working with Adrian, so I have this much overlap with him, so you can select whichever task that you want. And, oh, sorry. You can propose it. And the other person will… See an option to accept that. And once you accept it, an invite will be sent automatically at the correct time, along with the Zoom link for… which uses your personal Zoom link. That's pretty much it. I'll stop there.
- **22:06:17 · Nicklas Jensen:** All right, I'm next up here, and I am going to be sharing some porn. out of a conversation, actually, in a Zoom chat. So, one of the things on the meetup that happened was that we, uh… At least for a lot of the ducklings, we were freaking out about how much merch and how many things have been over the years. So the idea was to create a hall of fame for specifically T-shirts. So Miguel and I worked on that. For this, uh… For these hack days, the goal is you have a timeline, you can see all of the different designs that have been over the years. These are all the ones that Miguel could get a hold of so far. You can vote for your favorite ones. And also, if you don't feel like going through them all manually, you can just… pick out of sort of a random generated thing. And yeah, that's the T-shirt Hall of Fame. How do I stop sharing? Thank you.
- **22:07:30 · Dominik Kapusta:** All right, here's a demo of the… Long overdue feature on… macOS, which is the reader mode. Um… Reader mode. extracts text from websites. And here you can see that, uh, we're navigating to a website that, uh, is eligible for being displayed in reader mode, which contains relatively more text, uh, and, um… Just with a single button in the address bar, um… we turn on the reader mode, then we navigate away from that website, it just resets to a regular view. here, uh, you can see, uh, visiting another website, uh, that's also eligible for reader mode, and here's, uh, here's how it works, but it's not all, because I've worked on that, uh, during Hug Days, but. Um… a parallel to my work, actually. Marios was doing the same thing for iOS, so here it is, um… Here, the reader mode, uh… Button is in the website, uh… menu… And this demo will also show you that it respects the browser theme. So that, um… shows correctly in the dark theme as well. Even if you visit a website that… uh, doesn't honor the system theme, uh, then the reader mode, of course, uh, will be dark in this case, when the Guardian was in, uh, in light theme. Yeah, and this all was possible. to do thanks to a library by Meek that extracts. text from, from websites. So really the integration. It was fairly minimal here, and it was just a bit of work. That's why I said it was long overdue, because that library was published a few months ago already. That's it. Thank you.
- **22:10:01 · Valerie Kraucunas:** Alrighty, hello. Um, so, I'm Valerie. I work on the front-end team. Um, the thing I've been really heavily focused on all year is the rebrand, and that… with that has come… turning over, uh, style values from one… one way to a new way. Um… with that, we have these things called design tokens, which is basically, like, a codified design decision. And so, when we're styling… the new rebranded stuff we're using, or at least we want to be using all of these things. That said, um, a lot of times when you are handed designs that are rebranded. You end up with this scenario of… It'll be a mix of using the new style tokens and the old style tokens. So in this case, like the SDS styles are a legacy thing. Um, and I want to be very clear, this is not a shame. This is not a shame campaign or anything on designers, because oftentimes when they are building these things, they are… they have the old way in the same file as the. new way. And so it's really, really tedious to get down into every single nook and cranny and node or layer to make sure that you're using the new styles on everything and to make it even more complicated. So when developers see this, we just see this and we're like, why is this still here? Why are we still using SDS things? But when you get over into the design look, so this first column is when they're picking the first padding value, so space X07. When they're picking the second column value, space X03. They look the same. It's really easy to co-mingle the old style and the new. And so, um, basically my concept is to… be able to hand a Claude skill a Figma link, and have it go and traverse the entire. All child nodes of that, um… of whatever you linked over. So. I'm going to try this. I doubt it will… I doubt it'll work. It does not go as fast as I would like it to, but we'll give it a go. So, basically, what happens is you, you know, invoke it with the Figma link, then you give it an Asana link. And give it a name. Um… What that ends up spitting out is a nice report for designers to be able to go and update all the things that are using, uh, old values, or possibly things that are just out of system. As well, um, now I want to be clear, I'm standing on the shoulders of giants here, because, um, while we've been doing the SERP rebrand, uh, my partner in crime, Rachel Lim, created a… skill called Marlin Rebrando, which is basically a dictionary of, uh, knowledge for this was the old way, this is the new way, these are the things we should be checking for when we are executing the rebrand and the code. Um, so my skill is heavily relying on that, but, um… What it'll basically do is it will just… fully traverse like a node tree, if you will, all the way down. Um… And yeah, the things that we're looking for are the use of legacy variables. This is run from within the DDG. repo, for now, at least. So what it will do also is it'll look at the currently installed design token package and make sure that there aren't values being used in the designs that don't exist in. the installed package, then also looking for just colors that aren't tied to a system value, unselled text, et cetera. So what this ends up doing is it will create a. design token report, and. give the designers a link to the layer that. his, uh… has a property on it that's a problem, if you will, and then give it an idea of, like, this is an unexpected value, and then has an expected or a suggested value. The nice part about this as well is that developers. don't necessarily have to wait on designers to go. What?
- **22:14:28 · Julia Abreu:** Well… I'm so sorry, I hate to do it, but I have to interrupt you, because we're at 4 minutes now.
- **22:14:35 · Valerie Kraucunas:** Oh, shoot, I'm so sorry. Okay, well…
- **22:14:36 · Julia Abreu:** Sorry.
- **22:14:38 · Valerie Kraucunas:** Making everybody's life better, that's all.
- **22:14:50 · Stano:** Okay, I believe I was next. Um, I'm gonna… I hope you can see my screen now. Someone please confirm.
- **22:15:00 · Julia Abreu:** No, we cannot.
- **22:15:01 · Jason Ashby:** Nope.
- **22:15:02 · Stano:** You cannot? Let me try again. Where is… The sun got stuck.
- **22:15:24 · Julia Abreu:** Did you find the share button?
- **22:15:27 · Stano:** Give me a sec, I reconnected. You can take it out of my budget, I'll make it quick. It does not want to work. You can skip me. Go on.
- **22:15:45 · Anh Do:** Okay, looks like I'm next. So let me try to share my screen. Wow. Uh, how do I share, like, a… Particular windows, um… Sorry, um, I had so many windows opening at the moment. Oh, come on. Guys, it is not fun. Sorry. OK, I'm going to share my desktop then. One sec. Hopefully, you can see that on my screen. Can anyone see it? Okay. So for context, in OK, we are helping people in the US to remove personal information from data broker by submitting our request on their.
- **22:16:21 · Julia Abreu:** I can see it.
- **22:16:33 · Anh Do:** behalf. But recently, we have a lot of issues with broker block our archive with CloudFlare and CAPTCHA, so we can submit the form for the users. So this demo has been built based on Ben Smith design prototype. Let me play that.
@@ -19,6 +19,181 @@
"time": "21:03:03",
"speaker": "SS",
"text": "Hey, yes, um… I was recently doing some bug finding and fixing in the iOS app, uh, and had an opportunity to use the verify skill that Pete had put together for iOS. So the back… the background here is that, um. This is a skill that is used with PStack, if you remember Potato Mode. Who amongst us could forget? Uh, that is built around the concept of these verify skills, and Peter's made one for iOS that captures, like. recording of the screen, it captures, like, the pixels that were sent, like, everything you need to, like, prove that, you know, the agent has done the thing that it said it was going to do. Uh, so I used this for fixing a cr. evidence before the crash happening, after of it being fixed. It was super useful, worked great, so. Thank you, Pete."
},
{
"time": "21:03:50",
"speaker": "Jonathan Jackson",
"text": "Awesome. Thank you. Quick reminder that our next team retro is about four weeks from now. Just keep that in mind. It's coming up. It's going to come quickly, I think, for everybody this time of year. Um… we'll go through a quick round of rotation check-ins. Uh, this week for iOS Triage is Hasan, uh, followed by PCOR and then Yasek. Mac OS triage is Alessandro this week, followed by Ann and then Sabrina. For iOS release, it looks like whoever's on this week has closed the task by accident, but I hope you know who you are and you can bring it back for us. Next week will be Pao and then Hanyu. And for macOS…"
},
{
"time": "21:04:37",
"speaker": "Pete Smith",
"text": "Is that… sorry, Jonathan, was that iOS this week, was it?"
},
{
"time": "21:04:39",
"speaker": "Jonathan Jackson",
"text": "Yep. iOS release this week? Yeah, is that you?"
},
{
"time": "21:04:43",
"speaker": "Pete Smith",
"text": "Yeah, yeah, I closed it. It wasn't by accident, though. I just closed it because I have the other release tasks. But if you need it open, I can reopen."
},
{
"time": "21:04:47",
"speaker": "Jonathan Jackson",
"text": "Oh, okay. I mean, if we don't… no, we don't necessarily need it open. If it's done, it's done, but usually they've set open for the."
},
{
"time": "21:04:54",
"speaker": "Pete Smith",
"text": "Cool."
},
{
"time": "21:04:54",
"speaker": "Jonathan Jackson",
"text": "Doesn't matter. Uh, the important part is that you know you're doing it, you've dealt with it, which clearly you have. Uh, and then macOS release, uh, is Joseph this week, followed by Federico, and then Juan. Also, a reminder that we're starting to coordinate the winter holiday rotations. There's a task link in the meeting notes here, and I sent that out late last week. Maybe it was even early this week, early last week. We definitely need some folks to be able to cover for others who are out. A lot of those rotations land on people's out of office time. take a peek. If you can cover one of those, even if you're not normally on that rotation, and you're… but you're willing to, that would be lovely. It would help other people have the time off that they are looking for. If we get stuck, I'll get a little more involved once it sort of has run its course on basic conversation and then see what we can do to make sure this works for everybody. Um, but for now, start with if there's something that you're… either if you're on a rotation that you can't cover, mark yourself as unavailable. Most folks have already done that, but there's a few that still need to. Or if there's one you're willing to cover through the holiday, whether you're just around during that time or you're going to be out of office, but you're still willing to cover it, mark yourself as green, and then we'll go from there about other coverage. I see a couple of folks have already volunteered to cover a couple of weeks. I appreciate that. Time sensitive topics. Andre. An update on SmartLink from you. Thank you."
},
{
"time": "21:06:30",
"speaker": "Andre Martingo",
"text": "Yeah, just a quick follow-up on the previous update that we had. So, um, we have been running for, I think it's a second release now. Um, it showed up more mature, this one, so basically, uh, we had a lot of strings that were, uh. merge into master, so the PR for iOS and macOS are already merged. In fact, iOS, we went through already a specific flow because everything was not ready when we forked the release. So basically, we saw during this night. That was able to understand if the job was not completed and then wait for the job to be completed and pull all the strings. So we disable the danger rule in main. So basically now you can start merging untranslated stuff. and everything should work. Me and Pao, we are gonna still keep an eye to understand if something goes wrong. Uh, but for example, we already saw with Reedy, uh, another, uh, feature that we introduced, which is if you create a… If you merge into main and that job that got triggered on your commit will have some question from someone from SmartLink, it will create a task for you. So you don't need to always check if the job has any issues. proactively will create a task for you, if there is any issue. Um, so we expect everything to be much smoother, um, and basically, yeah, you can start merging, uh, English strings into mine."
},
{
"time": "21:08:05",
"speaker": "Jonathan Jackson",
"text": "Amazing. Thank you. And the automated task creation, if it has a question, is really excellent too because it closes that loop. Very cool. Dominic. Bitrise update. We're spending seven grand a month on UI tests."
},
{
"time": "21:08:25",
"speaker": "Dominik Kapusta",
"text": "Yes, thank you. On UI test on macOS 15 and 26 and 14. Which is now deleted. Um… That's what I learned. Oh. After switching to Bitrise, I kept asking Kiran about the costs of GitHub, expecting them to go down, and they went down to $7,000 a month. So I started digging, and I found out that this… Sort of… almost ideally matches the number of minutes we used on macOS 14, 15 and 26xlarge GitHub runners in October. And then, I also recalled that Bitrise rolled out an update to BuildHub pools, that they allow… when they allow to… scaling down pools to zero, essentially, so… You can create a pool of… Some machines, and if none of them is used. there are just zero machines in the pool, and it doesn't contribute to the overall capacity. So I thought, okay, I'm just gonna create pools for macOS 15 and macOS 26, and let's run UI tests in Bitrise, and it's live. As of now, it's in main, so all macOS UI tests now run on Bitrise machines, and we're aiming to remove that $7,000 a month from GitHub bill. Um, and in place of that, I'm proposing that we increase our Bitrise fleet, because we will need some more machines. in some time, especially when we start using Bitrise machines for Agentic workflows. But also, some queuing is happening already. So, I put together a very small, um… Um, proposal with a calculation. I only pinged Jonathan and Laura over there, for what it's worth. You can reach that proposal by going to that task and in a subtask. But it's essentially to just agree that we should pursue that at the very moment, because we are about to update the Bitrise deal. Jeff from Partnerships suggested that if we're… if we wanted to… if we are updating the deal at the moment, that's really to build cash, a separate thing, completely. Uh, if we're doing an amendment, we'll need legal review, we'll need some other reviews, probably, so… If we want to change something else, let's do it right now. To… to speed up the process and also save. Time and money of the legal team. So, that's that. The change for… for Apple team members is that UI tests will be… oh, there's actually one more change I forgot about. So the UI tests run on bitrise. And also… from now on, by default, UI tests will only run on macOS 27. If you want to run macOS… if you want to run macOS UI tests on older OS versions, you have to check the box. That's called all OS versions. It was previously on by default, now it's off by default. That's the change. But also, there's another news item I want to share. Alessandro is working on… an overhaul of how we trigger UI tests. So, we removed some triggers for UI tests workflows on pull requests, and in place of that, we'll be putting. UI test workflows triggered by a label on a pull request. Ale is working on that. We'll have more next week. Thank you."
},
{
"time": "21:12:21",
"speaker": "Jonathan Jackson",
"text": "One question on the UI test side. You talked about this for macOS only. Should we think about it for iOS as well, or is there a different story there already?"
},
{
"time": "21:12:29",
"speaker": "Dominik Kapusta",
"text": "It's a different story, because iOS doesn't need old OS versions, because it has simulators."
},
{
"time": "21:12:36",
"speaker": "Jonathan Jackson",
"text": "So those are already running in the rows."
},
{
"time": "21:12:36",
"speaker": "Dominik Kapusta",
"text": "It will be… Yeah, yeah."
},
{
"time": "21:12:40",
"speaker": "Jonathan Jackson",
"text": "Okay, awesome."
},
{
"time": "21:12:41",
"speaker": "Dominik Kapusta",
"text": "Thank you."
},
{
"time": "21:12:44",
"speaker": "Jonathan Jackson",
"text": "James, you're up next."
},
{
"time": "21:12:48",
"speaker": "JF",
"text": "Hey, um, this is just an update on something I started during the last hack days, which was… To look at automating our, uh… the pixel anomaly triage stage of our rotations, uh, initially starting with macOS, just because that's what I'm used to looking at on the rotation. I got it working reasonably well over half days, and then… When Jorge was on maintenance a few weeks ago, I manually ran it each day, um, kind of alongside him, just to see how it was going. Tuned it, made it less verbose over time. I think it's in a good place now. It's built on… There's a new GitHub Agentic workflow system that's been put in place by Lucas, which. Um, is now approved and ready to go, and it can… run on a GitHub workflow and access Asano, ClickHouse, um, you know, GitHub. info. So, basically, I feel like it's in a good place, so I'm going to enable it to run each day. Automatically, once the Pixel triage tasks are created for macOS, basically it annotates the existing description that gets put in the triage tasks. If it thinks anything needs further review, it will create a subtask and assign it to that person. Just like any of us would do if we were reviewing these things. So I'm going to enable it. We can see how it goes. I've dropped a link. In the, uh… chart both. dropped a document in the AI enablement, um… project with kind of how it works. I'll keep an eye on it as well as it runs and keep in touch with the people who are on rotations just to see what they think of it. And we can, you know, we can dial it in. But if anyone's got any, you know, questions or issues or. improvements they can suggest, then let me know. But I will turn that on shortly."
},
{
"time": "21:14:54",
"speaker": "Jonathan Jackson",
"text": "Very cool. Thank you. I'm going to ask you the same question I asked Dominic. What about iOS?"
},
{
"time": "21:14:59",
"speaker": "JF",
"text": "Yeah, I don't see why that should. should be an issue. It's the same kind of thing, but I just thought I'd start with one."
},
{
"time": "21:15:03",
"speaker": "Jonathan Jackson",
"text": "Yeah, I figured. Great. Yeah, yeah, sure. I just want to make sure that you're thinking ahead, as I imagine you are."
},
{
"time": "21:15:10",
"speaker": "JF",
"text": "Yeah."
},
{
"time": "21:15:11",
"speaker": "Jonathan Jackson",
"text": "Um, I guess I would say when you're, you know, once you're happy with it on macOS, you could turn it on for iOS. If you need more signal, you could turn it on for both even sooner, and you'd."
},
{
"time": "21:15:17",
"speaker": "JF",
"text": "Yeah. Sure."
},
{
"time": "21:15:21",
"speaker": "Jonathan Jackson",
"text": "But I'll leave that up to you to decide, obviously."
},
{
"time": "21:15:23",
"speaker": "JF",
"text": "Cool."
},
{
"time": "21:15:24",
"speaker": "Jonathan Jackson",
"text": "Thank you. Awesome. Okay. Miha, you're next."
},
{
"time": "21:15:33",
"speaker": "Michal Smaga",
"text": "Yeah, I wanted to do a short demo, like, to give you a privilege of, like, pre-release showcase of the feature we are working on. So, as you probably know, we have already ad blocking for YouTube, and now we are working on something called, like, sponsor blocking, or skipping sponsor segments, so… Uh, there may be, uh, like, some of the videos can have the… the parts where the creator, like, advertises some kind of, like, sponsored product, and sometimes you may want to skip those, and there's no easy way, unless you use some third-party, like, web extensions. And we are trying to leverage one of the solutions, which is SponsorBlock. So this is, like, both API and both web extension, and we try to, like, integrate it into our browsers, so… Uh, in the linked task, there's, like, a step-by-step, uh, description. So, uh, the current state is that we have the MVP, which is, like, very, like, uh, bare-bones, that, uh, there's no design, there's no copy, so there are no translations, so basically this is, like, a very bare-bones version. Of the future to validate if it works and if it, like, lives up to to our expectations. And this is like for internal use only, so it won't be shipped in that form. Definitely. So, you may already have it because it's enabled by the behind the internal user feature flag in this week's internal releases. So if you go into the task. There's, uh… There's a short description of the feature. There's the required version, which is, like, this week's internal. So either you need to use TestFlight for the apps versions, or if you have the direct download on macOS, it should be already on your desktop. And basically, the future works like that, that if you open, uh… video that contains a sponsored segment. I think this one does. You will see that there's, like, a highlight in the SIG bar, which is the sponsored segment, and… If you open the settings in ad blocking, you have the additional section here for the sponsored segments, where you can toggle between the manual and the automatic mode. So the default one is the manual one, so if you go into the section here, you can see there's, like, a button. You can click it, and it goes to the end of the section, and, it plays back. In the automatic, it automatically, like, skips the segment when it's being encountered. Uh, yeah, and this is basically it, and should just work for everyone, so the goal is to have it for internal users, for the employees, be enabled by default. there will be wider announcement, uh, for… for it, uh, and probably we'll have some kind of, like, channel to gather the feedback, so keep an eye on… on that, and yeah, if you want, like, sponsored segment-free YouTube experience, you'll… Feel free to, like, use it and enjoy it. If you have any open questions, I'm… Sure. Thank you."
},
{
"time": "21:18:56",
"speaker": "Jonathan Jackson",
"text": "Very cool, thank you. I'm definitely gonna check that one out."
},
{
"time": "21:18:59",
"speaker": "Michal Smaga",
"text": "I see there was like a question in the chat. Yeah, Brindy asked about if the YouTubers are getting paid. Yeah, so this is like not the ads that are served by YouTube. So this is like part of the video, which is the tricky part. And, uh, like, from all the information that we were able to gather, uh, they are, like, probably paid upfront, so there's, like, external contract that they are, and there's no indication on. the video, or, like, specific segment, I'm not sure if there are, like, analytics for that, uh, that if given, like, sponsored segment was being, uh, watched. Uh, and, uh, yeah, so I don't think so, and basically the data is not extracted from YouTube, it's basically, like, from a third party, so the sponsor block is, like, the API, where… which is, like, open source, and, like. community-driven, so, like, people, like, submit the segments, uh, for the data, and yeah, we can use that."
},
{
"time": "21:20:03",
"speaker": "Jonathan Jackson",
"text": "Cool. Thank you. Yeah, that was a good question. Uh, Brenda, your revenue stream sounds like it's safe. Um… Does anybody have anything else to add? Any questions, demos, other things you want to talk about today?"
},
{
"time": "21:20:18",
"speaker": "Jorge Leandro Perez",
"text": "Question, do we have a meetup location defined yet, or not yet?"
},
{
"time": "21:20:23",
"speaker": "Jonathan Jackson",
"text": "Uh, location is not defined the way… there are… the process is a little different this year, um, and John B. has taken over the… the week questionnaire, and sort of getting those weeks figured out. I think you probably have seen that. Um… I need to check back in with him now that that's happened, and figure out what next steps are in terms of location planning. He's been in touch with us a little bit, um, but that'll be, I think, probably the next step, so stay tuned."
},
{
"time": "21:20:50",
"speaker": "Jorge Leandro Perez",
"text": "Thank you."
},
{
"time": "21:20:58",
"speaker": "Jonathan Jackson",
"text": "Thank you. Anything else? Anybody? OK. Thank you all. It's good to see you. Have a great day."
}
]
}
@@ -14,3 +14,38 @@ source: "https://us01mynotes.zoom.us/VSuzeqZCR5O2z5KtmaQt3g"
- **21:01:48 · Jorge Leandro Perez:** I'm usually the one who can't stand the silence in elevators, so I start talking.
- **21:01:55 · Jonathan Jackson:** I appreciate that you haven't decided to bite Muzak into our meeting, sir, so… All right. Why don't we go ahead and get started? Sam, you want to kick us off with your appreciation?
- **21:03:03 · SS:** Hey, yes, um… I was recently doing some bug finding and fixing in the iOS app, uh, and had an opportunity to use the verify skill that Pete had put together for iOS. So the back… the background here is that, um. This is a skill that is used with PStack, if you remember Potato Mode. Who amongst us could forget? Uh, that is built around the concept of these verify skills, and Peter's made one for iOS that captures, like. recording of the screen, it captures, like, the pixels that were sent, like, everything you need to, like, prove that, you know, the agent has done the thing that it said it was going to do. Uh, so I used this for fixing a cr. evidence before the crash happening, after of it being fixed. It was super useful, worked great, so. Thank you, Pete.
- **21:03:50 · Jonathan Jackson:** Awesome. Thank you. Quick reminder that our next team retro is about four weeks from now. Just keep that in mind. It's coming up. It's going to come quickly, I think, for everybody this time of year. Um… we'll go through a quick round of rotation check-ins. Uh, this week for iOS Triage is Hasan, uh, followed by PCOR and then Yasek. Mac OS triage is Alessandro this week, followed by Ann and then Sabrina. For iOS release, it looks like whoever's on this week has closed the task by accident, but I hope you know who you are and you can bring it back for us. Next week will be Pao and then Hanyu. And for macOS…
- **21:04:37 · Pete Smith:** Is that… sorry, Jonathan, was that iOS this week, was it?
- **21:04:39 · Jonathan Jackson:** Yep. iOS release this week? Yeah, is that you?
- **21:04:43 · Pete Smith:** Yeah, yeah, I closed it. It wasn't by accident, though. I just closed it because I have the other release tasks. But if you need it open, I can reopen.
- **21:04:47 · Jonathan Jackson:** Oh, okay. I mean, if we don't… no, we don't necessarily need it open. If it's done, it's done, but usually they've set open for the.
- **21:04:54 · Pete Smith:** Cool.
- **21:04:54 · Jonathan Jackson:** Doesn't matter. Uh, the important part is that you know you're doing it, you've dealt with it, which clearly you have. Uh, and then macOS release, uh, is Joseph this week, followed by Federico, and then Juan. Also, a reminder that we're starting to coordinate the winter holiday rotations. There's a task link in the meeting notes here, and I sent that out late last week. Maybe it was even early this week, early last week. We definitely need some folks to be able to cover for others who are out. A lot of those rotations land on people's out of office time. take a peek. If you can cover one of those, even if you're not normally on that rotation, and you're… but you're willing to, that would be lovely. It would help other people have the time off that they are looking for. If we get stuck, I'll get a little more involved once it sort of has run its course on basic conversation and then see what we can do to make sure this works for everybody. Um, but for now, start with if there's something that you're… either if you're on a rotation that you can't cover, mark yourself as unavailable. Most folks have already done that, but there's a few that still need to. Or if there's one you're willing to cover through the holiday, whether you're just around during that time or you're going to be out of office, but you're still willing to cover it, mark yourself as green, and then we'll go from there about other coverage. I see a couple of folks have already volunteered to cover a couple of weeks. I appreciate that. Time sensitive topics. Andre. An update on SmartLink from you. Thank you.
- **21:06:30 · Andre Martingo:** Yeah, just a quick follow-up on the previous update that we had. So, um, we have been running for, I think it's a second release now. Um, it showed up more mature, this one, so basically, uh, we had a lot of strings that were, uh. merge into master, so the PR for iOS and macOS are already merged. In fact, iOS, we went through already a specific flow because everything was not ready when we forked the release. So basically, we saw during this night. That was able to understand if the job was not completed and then wait for the job to be completed and pull all the strings. So we disable the danger rule in main. So basically now you can start merging untranslated stuff. and everything should work. Me and Pao, we are gonna still keep an eye to understand if something goes wrong. Uh, but for example, we already saw with Reedy, uh, another, uh, feature that we introduced, which is if you create a… If you merge into main and that job that got triggered on your commit will have some question from someone from SmartLink, it will create a task for you. So you don't need to always check if the job has any issues. proactively will create a task for you, if there is any issue. Um, so we expect everything to be much smoother, um, and basically, yeah, you can start merging, uh, English strings into mine.
- **21:08:05 · Jonathan Jackson:** Amazing. Thank you. And the automated task creation, if it has a question, is really excellent too because it closes that loop. Very cool. Dominic. Bitrise update. We're spending seven grand a month on UI tests.
- **21:08:25 · Dominik Kapusta:** Yes, thank you. On UI test on macOS 15 and 26 and 14. Which is now deleted. Um… That's what I learned. Oh. After switching to Bitrise, I kept asking Kiran about the costs of GitHub, expecting them to go down, and they went down to $7,000 a month. So I started digging, and I found out that this… Sort of… almost ideally matches the number of minutes we used on macOS 14, 15 and 26xlarge GitHub runners in October. And then, I also recalled that Bitrise rolled out an update to BuildHub pools, that they allow… when they allow to… scaling down pools to zero, essentially, so… You can create a pool of… Some machines, and if none of them is used. there are just zero machines in the pool, and it doesn't contribute to the overall capacity. So I thought, okay, I'm just gonna create pools for macOS 15 and macOS 26, and let's run UI tests in Bitrise, and it's live. As of now, it's in main, so all macOS UI tests now run on Bitrise machines, and we're aiming to remove that $7,000 a month from GitHub bill. Um, and in place of that, I'm proposing that we increase our Bitrise fleet, because we will need some more machines. in some time, especially when we start using Bitrise machines for Agentic workflows. But also, some queuing is happening already. So, I put together a very small, um… Um, proposal with a calculation. I only pinged Jonathan and Laura over there, for what it's worth. You can reach that proposal by going to that task and in a subtask. But it's essentially to just agree that we should pursue that at the very moment, because we are about to update the Bitrise deal. Jeff from Partnerships suggested that if we're… if we wanted to… if we are updating the deal at the moment, that's really to build cash, a separate thing, completely. Uh, if we're doing an amendment, we'll need legal review, we'll need some other reviews, probably, so… If we want to change something else, let's do it right now. To… to speed up the process and also save. Time and money of the legal team. So, that's that. The change for… for Apple team members is that UI tests will be… oh, there's actually one more change I forgot about. So the UI tests run on bitrise. And also… from now on, by default, UI tests will only run on macOS 27. If you want to run macOS… if you want to run macOS UI tests on older OS versions, you have to check the box. That's called all OS versions. It was previously on by default, now it's off by default. That's the change. But also, there's another news item I want to share. Alessandro is working on… an overhaul of how we trigger UI tests. So, we removed some triggers for UI tests workflows on pull requests, and in place of that, we'll be putting. UI test workflows triggered by a label on a pull request. Ale is working on that. We'll have more next week. Thank you.
- **21:12:21 · Jonathan Jackson:** One question on the UI test side. You talked about this for macOS only. Should we think about it for iOS as well, or is there a different story there already?
- **21:12:29 · Dominik Kapusta:** It's a different story, because iOS doesn't need old OS versions, because it has simulators.
- **21:12:36 · Jonathan Jackson:** So those are already running in the rows.
- **21:12:36 · Dominik Kapusta:** It will be… Yeah, yeah.
- **21:12:40 · Jonathan Jackson:** Okay, awesome.
- **21:12:41 · Dominik Kapusta:** Thank you.
- **21:12:44 · Jonathan Jackson:** James, you're up next.
- **21:12:48 · JF:** Hey, um, this is just an update on something I started during the last hack days, which was… To look at automating our, uh… the pixel anomaly triage stage of our rotations, uh, initially starting with macOS, just because that's what I'm used to looking at on the rotation. I got it working reasonably well over half days, and then… When Jorge was on maintenance a few weeks ago, I manually ran it each day, um, kind of alongside him, just to see how it was going. Tuned it, made it less verbose over time. I think it's in a good place now. It's built on… There's a new GitHub Agentic workflow system that's been put in place by Lucas, which. Um, is now approved and ready to go, and it can… run on a GitHub workflow and access Asano, ClickHouse, um, you know, GitHub. info. So, basically, I feel like it's in a good place, so I'm going to enable it to run each day. Automatically, once the Pixel triage tasks are created for macOS, basically it annotates the existing description that gets put in the triage tasks. If it thinks anything needs further review, it will create a subtask and assign it to that person. Just like any of us would do if we were reviewing these things. So I'm going to enable it. We can see how it goes. I've dropped a link. In the, uh… chart both. dropped a document in the AI enablement, um… project with kind of how it works. I'll keep an eye on it as well as it runs and keep in touch with the people who are on rotations just to see what they think of it. And we can, you know, we can dial it in. But if anyone's got any, you know, questions or issues or. improvements they can suggest, then let me know. But I will turn that on shortly.
- **21:14:54 · Jonathan Jackson:** Very cool. Thank you. I'm going to ask you the same question I asked Dominic. What about iOS?
- **21:14:59 · JF:** Yeah, I don't see why that should. should be an issue. It's the same kind of thing, but I just thought I'd start with one.
- **21:15:03 · Jonathan Jackson:** Yeah, I figured. Great. Yeah, yeah, sure. I just want to make sure that you're thinking ahead, as I imagine you are.
- **21:15:10 · JF:** Yeah.
- **21:15:11 · Jonathan Jackson:** Um, I guess I would say when you're, you know, once you're happy with it on macOS, you could turn it on for iOS. If you need more signal, you could turn it on for both even sooner, and you'd.
- **21:15:17 · JF:** Yeah. Sure.
- **21:15:21 · Jonathan Jackson:** But I'll leave that up to you to decide, obviously.
- **21:15:23 · JF:** Cool.
- **21:15:24 · Jonathan Jackson:** Thank you. Awesome. Okay. Miha, you're next.
- **21:15:33 · Michal Smaga:** Yeah, I wanted to do a short demo, like, to give you a privilege of, like, pre-release showcase of the feature we are working on. So, as you probably know, we have already ad blocking for YouTube, and now we are working on something called, like, sponsor blocking, or skipping sponsor segments, so… Uh, there may be, uh, like, some of the videos can have the… the parts where the creator, like, advertises some kind of, like, sponsored product, and sometimes you may want to skip those, and there's no easy way, unless you use some third-party, like, web extensions. And we are trying to leverage one of the solutions, which is SponsorBlock. So this is, like, both API and both web extension, and we try to, like, integrate it into our browsers, so… Uh, in the linked task, there's, like, a step-by-step, uh, description. So, uh, the current state is that we have the MVP, which is, like, very, like, uh, bare-bones, that, uh, there's no design, there's no copy, so there are no translations, so basically this is, like, a very bare-bones version. Of the future to validate if it works and if it, like, lives up to to our expectations. And this is like for internal use only, so it won't be shipped in that form. Definitely. So, you may already have it because it's enabled by the behind the internal user feature flag in this week's internal releases. So if you go into the task. There's, uh… There's a short description of the feature. There's the required version, which is, like, this week's internal. So either you need to use TestFlight for the apps versions, or if you have the direct download on macOS, it should be already on your desktop. And basically, the future works like that, that if you open, uh… video that contains a sponsored segment. I think this one does. You will see that there's, like, a highlight in the SIG bar, which is the sponsored segment, and… If you open the settings in ad blocking, you have the additional section here for the sponsored segments, where you can toggle between the manual and the automatic mode. So the default one is the manual one, so if you go into the section here, you can see there's, like, a button. You can click it, and it goes to the end of the section, and, it plays back. In the automatic, it automatically, like, skips the segment when it's being encountered. Uh, yeah, and this is basically it, and should just work for everyone, so the goal is to have it for internal users, for the employees, be enabled by default. there will be wider announcement, uh, for… for it, uh, and probably we'll have some kind of, like, channel to gather the feedback, so keep an eye on… on that, and yeah, if you want, like, sponsored segment-free YouTube experience, you'll… Feel free to, like, use it and enjoy it. If you have any open questions, I'm… Sure. Thank you.
- **21:18:56 · Jonathan Jackson:** Very cool, thank you. I'm definitely gonna check that one out.
- **21:18:59 · Michal Smaga:** I see there was like a question in the chat. Yeah, Brindy asked about if the YouTubers are getting paid. Yeah, so this is like not the ads that are served by YouTube. So this is like part of the video, which is the tricky part. And, uh, like, from all the information that we were able to gather, uh, they are, like, probably paid upfront, so there's, like, external contract that they are, and there's no indication on. the video, or, like, specific segment, I'm not sure if there are, like, analytics for that, uh, that if given, like, sponsored segment was being, uh, watched. Uh, and, uh, yeah, so I don't think so, and basically the data is not extracted from YouTube, it's basically, like, from a third party, so the sponsor block is, like, the API, where… which is, like, open source, and, like. community-driven, so, like, people, like, submit the segments, uh, for the data, and yeah, we can use that.
- **21:20:03 · Jonathan Jackson:** Cool. Thank you. Yeah, that was a good question. Uh, Brenda, your revenue stream sounds like it's safe. Um… Does anybody have anything else to add? Any questions, demos, other things you want to talk about today?
- **21:20:18 · Jorge Leandro Perez:** Question, do we have a meetup location defined yet, or not yet?
- **21:20:23 · Jonathan Jackson:** Uh, location is not defined the way… there are… the process is a little different this year, um, and John B. has taken over the… the week questionnaire, and sort of getting those weeks figured out. I think you probably have seen that. Um… I need to check back in with him now that that's happened, and figure out what next steps are in terms of location planning. He's been in touch with us a little bit, um, but that'll be, I think, probably the next step, so stay tuned.
- **21:20:50 · Jorge Leandro Perez:** Thank you.
- **21:20:58 · Jonathan Jackson:** Thank you. Anything else? Anybody? OK. Thank you all. It's good to see you. Have a great day.
@@ -0,0 +1,124 @@
{
"schema_version": 1,
"note_id": "7JZgnceAT1aZ2H75ok4dmA",
"title": "Costas Brammer's Personal Meeting Room",
"meeting_date": null,
"source_url": "https://us01mynotes.zoom.us/7JZgnceAT1aZ2H75ok4dmA",
"transcript": [
{
"time": "21:03:34",
"speaker": "Costas Brammer",
"text": "All right. Go on. Whoever talks can quickly choose the next person. Pick a person. We'll be quick."
},
{
"time": "21:03:40",
"speaker": "Juan Diana",
"text": "Okay, Tom."
},
{
"time": "21:03:43",
"speaker": "Tom Strba",
"text": "To be honest, I can't think of anything, but I am very interested in… interested in 3D printing, and I would like to buy a 3D printer. There is a lot of very useful stuff you can print. If you haven't checked, check on YouTube what people print from it, and I think I am definitely… buying that. Lukasz, how about you?"
},
{
"time": "21:04:08",
"speaker": "LP",
"text": "Well, well, well, I actually bought a 3D printer. So that's a cheat answer, because I can print all the other cool stuff that I want run. So I just started out. I haven't really actually printed much, just like one. phone stand that I needed for our, kind of, baby monitor, and I'm printing… after this call, I'm gonna go load some filament to print, like, a storage box for the shower, because, like, all of the razor blades are just… running around, or, like, being thrown around, so I'm gonna be printing that, but my excuse for printing it was that I had needed to have some covers for the pendant lamps, and obviously, like, I can't buy those, right? I need to print them, so yeah, that was my excuse, and we'll see if the wife is gonna be satisfied after a longer period of usage. All right, so I'm going to pass it over to Alex."
},
{
"time": "21:05:04",
"speaker": "Alex Martemyanov",
"text": "Sorry I missed the question."
},
{
"time": "21:05:07",
"speaker": "Costas Brammer",
"text": "Yeah, it's, what's the, what's the, I guess the best thing you've bought recently? Just a quick, quick question."
},
{
"time": "21:05:14",
"speaker": "Alex Martemyanov",
"text": "Best thing I bought recently. Wow, that's a good question."
},
{
"time": "21:05:19",
"speaker": "Costas Brammer",
"text": "Don't overthink it, Alex. It could be a bowl of cereal, that's."
},
{
"time": "21:05:26",
"speaker": "Alex Martemyanov",
"text": "I don't know, maybe some local storage, and it's mostly about setting up some… some stuff up to my automation and using AI, like selective backup for my MacBook, especially having problems with 2TB drive getting filled really quickly with all this. Virtual machines and containers and all this stuff. Oh. Yeah, maybe some. gaming and, uh, Steam, uh, Steam games, uh, so I installed it. SteamOS and my older PC. And it's… surprisingly, it's 10 years old, but it's still pretty good, like i7, 32GB, and it runs a lot of games, pretty smooth, and I try to… Play it with my daughter. Yes, that would be it. From… Oh, that's there. Zach."
},
{
"time": "21:06:36",
"speaker": "ZE",
"text": "Um, so I'll, like, break the rules a little bit. So, it's less of a purchase and more like, uh… well, I guess it's something I spend money on. So, basically, like, my, like… I've got one of these credit cards where you, like, earn points or whatever, and it's got, like, a ridiculous. yearly fee, but you give all these benefits that can make it worth it, and so, like, this week, I've just been, like, running through, like, all the benefits, like, booking trips, like, family things, doing this, just, like, literally going through, trying to, like, cash in on all the. all, like, the… the little things that you're supposed to get, and so, uh… yeah, I feel very… I feel very accomplished, uh, in doing that. Like, I'm really, uh… Like I'm really getting the better of the creditors. So, which is exactly what they want me to think."
},
{
"time": "21:07:26",
"speaker": "Juan Diana",
"text": "But you're spending more!"
},
{
"time": "21:07:27",
"speaker": "ZE",
"text": "Well, no, I'm like, it's like, I'm using just the amount that… I'm spending just the amount that you get back. Yeah, yes. But, like, yeah, I'm using the credit card. I mean, presumably they get all that stuff for next to nothing in partnerships."
},
{
"time": "21:07:34",
"speaker": "Juan Diana",
"text": "The right amount. Okay."
},
{
"time": "21:07:51",
"speaker": "ZE",
"text": "Oh, I'm supposed to assign it, sorry. Bobby."
},
{
"time": "21:07:55",
"speaker": "Bobby (he/him)",
"text": "Uh, let's see. I was… I don't… I haven't really bought anything that exciting recently, but I'm gonna go back a few months. It was inspired by your next ad, Zach, which is… for the Knicks finals, which is now a few months ago, I live in New York City. I'm not a Knicks fan, but I decided I was gonna become one for the finals, and so to watch the finals, I went to my buddy's roof. And we set up a projector, but we needed a battery, because there's no plug up there. So I got this really small, like, mobile battery pack that I had to get really high wattage, because it needed to power a projector. And lo and behold, it is the perfect solution for me to work from anywhere with my laptop, because I didn't know there were mobile battery packs that have high enough wattage that I can just, like, use them. So now I've been working in the park, I've been working all over the place, because I can keep my all-day battery going, um, with my mobile battery pack. So high wattage mobile battery pack is my answer to this question, I guess."
},
{
"time": "21:08:55",
"speaker": "Chuck (he/him)",
"text": "I'm gonna jump in here, Bobby, to pitch this specific one from Anchor."
},
{
"time": "21:08:58",
"speaker": "Bobby (he/him)",
"text": "Yeah, go for it."
},
{
"time": "21:09:02",
"speaker": "Chuck (he/him)",
"text": "which has a wireless charger on the bottom. And I… Get out of here. But it has a charging base for the wireless charger that also doubles as basically a tiny power strip. Um, so that it magnetically holds here while it's charging, and has several other charges in it, so you can charge other things at once. It makes an absolutely incredible little package of… low weight battery charging. Goodness. Big recommend. Anchor's generally awesome, but now I'll… Zach. All right, that'll help do my real answer. Earplugs. Tiny, tiny little earplugs that go into a little case, it's… bits of my palm. I have three young kids, and I can't just put earplugs in, can't put my head, my AirPods in, and just go on transparency mode when they are overstimulating, which is about three. But these little earbuds, they're really nice. It's super flush to your head, um, and they don't actually block sound. They just have, like, a little aperture that restrains the amount of sound that gets through to your ear, and it's tunable. So, depending on how loud they are or where they are, I can just shrink the little aperture and make… narrow the amount of sound that's going through and feel… slightly recapture a small amount of sanity, which is completely priceless. Um… Bucky, I don't think you've gone yet."
},
{
"time": "21:10:24",
"speaker": "BE",
"text": "No, I haven't. You're gonna have to send me the link to those AAA bots, by the way. Chuck. Take a cup of Hadley. Uh, for me, it's, uh… a tie between… I had a delightful salmon and cream cheese sandwich yesterday. I'm on quite a strict diet, because I've been training, um, for High Rocks, but… that was one of the rare occasions where I veered off my diet. And, uh… or, sorry, a walking pad that I bought last week, but I haven't actually used it yet, so… I don't necessarily know if that could take top spot, but it was my, um… I suppose the more recent of a relatively. Large purchase that I can remember anyway. Uh… shit. She's left. Miguel, have you gone?"
},
{
"time": "21:11:19",
"speaker": "Miguel",
"text": "Sort of, but I can share one more thing. I recently, no, so I restarted because I had a salt shotgun, but this is a little ESP32 like board. It has a small touch screen. It's like two inches. Cool thing is I have some like home IoT stuff."
},
{
"time": "21:11:21",
"speaker": "BE",
"text": "Oh!"
},
{
"time": "21:11:36",
"speaker": "Miguel",
"text": "So this particular thing I bought because my kid has decided lately that he doesn't like sleeping. So he'll wake up like 3 a.m. and say, I no longer sleep in tonight. So the point is I'm building a UI so it's literally he can tap it if it's between 7 and 7. It'll show a moon. After 7:00 AM, it'll show a sun. So sort of a trainer. So again, he can look at it. It's like, OK, not yet. It's a little cool thing I can program in this. Yeah. Sort of build on top of. Uh, Danilo?"
},
{
"time": "21:12:09",
"speaker": "D",
"text": "Yeah. Genius to be on this again. When it comes to me, I guess it's automatic coffee grinder. Yeah. I used to love manual coffee grinder, and it was my kind of morning ritual when I was grinding coffee, pouring it for myself, but recently we realized that we drink more coffee."
}
]
}
@@ -0,0 +1,36 @@
---
title: "Costas Brammer's Personal Meeting Room"
date: null
zoom_note_id: "7JZgnceAT1aZ2H75ok4dmA"
source: "https://us01mynotes.zoom.us/7JZgnceAT1aZ2H75ok4dmA"
---
# Costas Brammer's Personal Meeting Room
[Zoom My Notes](https://us01mynotes.zoom.us/7JZgnceAT1aZ2H75ok4dmA)
## Transcript
- **21:03:34 · Costas Brammer:** All right. Go on. Whoever talks can quickly choose the next person. Pick a person. We'll be quick.
- **21:03:40 · Juan Diana:** Okay, Tom.
- **21:03:43 · Tom Strba:** To be honest, I can't think of anything, but I am very interested in… interested in 3D printing, and I would like to buy a 3D printer. There is a lot of very useful stuff you can print. If you haven't checked, check on YouTube what people print from it, and I think I am definitely… buying that. Lukasz, how about you?
- **21:04:08 · LP:** Well, well, well, I actually bought a 3D printer. So that's a cheat answer, because I can print all the other cool stuff that I want run. So I just started out. I haven't really actually printed much, just like one. phone stand that I needed for our, kind of, baby monitor, and I'm printing… after this call, I'm gonna go load some filament to print, like, a storage box for the shower, because, like, all of the razor blades are just… running around, or, like, being thrown around, so I'm gonna be printing that, but my excuse for printing it was that I had needed to have some covers for the pendant lamps, and obviously, like, I can't buy those, right? I need to print them, so yeah, that was my excuse, and we'll see if the wife is gonna be satisfied after a longer period of usage. All right, so I'm going to pass it over to Alex.
- **21:05:04 · Alex Martemyanov:** Sorry I missed the question.
- **21:05:07 · Costas Brammer:** Yeah, it's, what's the, what's the, I guess the best thing you've bought recently? Just a quick, quick question.
- **21:05:14 · Alex Martemyanov:** Best thing I bought recently. Wow, that's a good question.
- **21:05:19 · Costas Brammer:** Don't overthink it, Alex. It could be a bowl of cereal, that's.
- **21:05:26 · Alex Martemyanov:** I don't know, maybe some local storage, and it's mostly about setting up some… some stuff up to my automation and using AI, like selective backup for my MacBook, especially having problems with 2TB drive getting filled really quickly with all this. Virtual machines and containers and all this stuff. Oh. Yeah, maybe some. gaming and, uh, Steam, uh, Steam games, uh, so I installed it. SteamOS and my older PC. And it's… surprisingly, it's 10 years old, but it's still pretty good, like i7, 32GB, and it runs a lot of games, pretty smooth, and I try to… Play it with my daughter. Yes, that would be it. From… Oh, that's there. Zach.
- **21:06:36 · ZE:** Um, so I'll, like, break the rules a little bit. So, it's less of a purchase and more like, uh… well, I guess it's something I spend money on. So, basically, like, my, like… I've got one of these credit cards where you, like, earn points or whatever, and it's got, like, a ridiculous. yearly fee, but you give all these benefits that can make it worth it, and so, like, this week, I've just been, like, running through, like, all the benefits, like, booking trips, like, family things, doing this, just, like, literally going through, trying to, like, cash in on all the. all, like, the… the little things that you're supposed to get, and so, uh… yeah, I feel very… I feel very accomplished, uh, in doing that. Like, I'm really, uh… Like I'm really getting the better of the creditors. So, which is exactly what they want me to think.
- **21:07:26 · Juan Diana:** But you're spending more!
- **21:07:27 · ZE:** Well, no, I'm like, it's like, I'm using just the amount that… I'm spending just the amount that you get back. Yeah, yes. But, like, yeah, I'm using the credit card. I mean, presumably they get all that stuff for next to nothing in partnerships.
- **21:07:34 · Juan Diana:** The right amount. Okay.
- **21:07:51 · ZE:** Oh, I'm supposed to assign it, sorry. Bobby.
- **21:07:55 · Bobby (he/him):** Uh, let's see. I was… I don't… I haven't really bought anything that exciting recently, but I'm gonna go back a few months. It was inspired by your next ad, Zach, which is… for the Knicks finals, which is now a few months ago, I live in New York City. I'm not a Knicks fan, but I decided I was gonna become one for the finals, and so to watch the finals, I went to my buddy's roof. And we set up a projector, but we needed a battery, because there's no plug up there. So I got this really small, like, mobile battery pack that I had to get really high wattage, because it needed to power a projector. And lo and behold, it is the perfect solution for me to work from anywhere with my laptop, because I didn't know there were mobile battery packs that have high enough wattage that I can just, like, use them. So now I've been working in the park, I've been working all over the place, because I can keep my all-day battery going, um, with my mobile battery pack. So high wattage mobile battery pack is my answer to this question, I guess.
- **21:08:55 · Chuck (he/him):** I'm gonna jump in here, Bobby, to pitch this specific one from Anchor.
- **21:08:58 · Bobby (he/him):** Yeah, go for it.
- **21:09:02 · Chuck (he/him):** which has a wireless charger on the bottom. And I… Get out of here. But it has a charging base for the wireless charger that also doubles as basically a tiny power strip. Um, so that it magnetically holds here while it's charging, and has several other charges in it, so you can charge other things at once. It makes an absolutely incredible little package of… low weight battery charging. Goodness. Big recommend. Anchor's generally awesome, but now I'll… Zach. All right, that'll help do my real answer. Earplugs. Tiny, tiny little earplugs that go into a little case, it's… bits of my palm. I have three young kids, and I can't just put earplugs in, can't put my head, my AirPods in, and just go on transparency mode when they are overstimulating, which is about three. But these little earbuds, they're really nice. It's super flush to your head, um, and they don't actually block sound. They just have, like, a little aperture that restrains the amount of sound that gets through to your ear, and it's tunable. So, depending on how loud they are or where they are, I can just shrink the little aperture and make… narrow the amount of sound that's going through and feel… slightly recapture a small amount of sanity, which is completely priceless. Um… Bucky, I don't think you've gone yet.
- **21:10:24 · BE:** No, I haven't. You're gonna have to send me the link to those AAA bots, by the way. Chuck. Take a cup of Hadley. Uh, for me, it's, uh… a tie between… I had a delightful salmon and cream cheese sandwich yesterday. I'm on quite a strict diet, because I've been training, um, for High Rocks, but… that was one of the rare occasions where I veered off my diet. And, uh… or, sorry, a walking pad that I bought last week, but I haven't actually used it yet, so… I don't necessarily know if that could take top spot, but it was my, um… I suppose the more recent of a relatively. Large purchase that I can remember anyway. Uh… shit. She's left. Miguel, have you gone?
- **21:11:19 · Miguel:** Sort of, but I can share one more thing. I recently, no, so I restarted because I had a salt shotgun, but this is a little ESP32 like board. It has a small touch screen. It's like two inches. Cool thing is I have some like home IoT stuff.
- **21:11:21 · BE:** Oh!
- **21:11:36 · Miguel:** So this particular thing I bought because my kid has decided lately that he doesn't like sleeping. So he'll wake up like 3 a.m. and say, I no longer sleep in tonight. So the point is I'm building a UI so it's literally he can tap it if it's between 7 and 7. It'll show a moon. After 7:00 AM, it'll show a sun. So sort of a trainer. So again, he can look at it. It's like, OK, not yet. It's a little cool thing I can program in this. Yeah. Sort of build on top of. Uh, Danilo?
- **21:12:09 · D:** Yeah. Genius to be on this again. When it comes to me, I guess it's automatic coffee grinder. Yeah. I used to love manual coffee grinder, and it was my kind of morning ritual when I was grinding coffee, pouring it for myself, but recently we realized that we drink more coffee.
@@ -19,6 +19,641 @@
"time": "16:02:10",
"speaker": "Dominik Kapusta",
"text": "Jest."
},
{
"time": "16:02:13",
"speaker": "Alex Martemyanov",
"text": "And basically no progress on other stuff yet."
},
{
"time": "16:02:19",
"speaker": "Alex Martemyanov",
"text": "I need to finish the priority cleanup. So basically last week we've discussed it during the OE that we should. Basically wrap up the project with the. No visible outcomes. There was a slight decline in the numbers and several genuine bugs fixed, but they were not representing the main feature breakage."
},
{
"time": "16:02:45",
"speaker": "Dominik Kapusta",
"text": "Nie. Drop, Ios."
},
{
"time": "16:02:50",
"speaker": "Alex Martemyanov",
"text": "I know."
},
{
"time": "16:02:57",
"speaker": "Dominik Kapusta",
"text": "Łazać."
},
{
"time": "16:03:00",
"speaker": "Alex Martemyanov",
"text": "There was some decline, maybe a third. So there was a decline in some part. of the numbers, but there is still around 40%. of the event sequence of the received events that so basically most of it is. The data we received for most of the breakage pixels represents just regular normal flow and nothing obviously."
},
{
"time": "16:03:34",
"speaker": "Dominik Kapusta",
"text": "Bardzo."
},
{
"time": "16:03:34",
"speaker": "Alex Martemyanov",
"text": "pointing to anything. So the discussion during the OE was to basically clean up the inefficient fixes that we've got. The feature flags for the features that are removed and some. Um, like the pinned tabs management fix, uh, that are, um, staying without the feature flag. And, uh… Yeah, clean up the data that's being collected, because we've collected enough of it, and, uh, implement some permanent data collection that stays there for, uh. for diagnostics for the future possible breakages. And try to add some in. inside of the extension, some metrics inside of the extension to, um, to track the loading progress, basically. Where we… where do we stop, and where the messaging."
},
{
"time": "16:04:23",
"speaker": "Dominik Kapusta",
"text": "Ok."
},
{
"time": "16:04:31",
"speaker": "Alex Martemyanov",
"text": "Stops. and try to reason about it from from that perspective."
},
{
"time": "16:04:37",
"speaker": "Dominik Kapusta",
"text": "Misji."
},
{
"time": "16:04:39",
"speaker": "Alex Martemyanov",
"text": "So, uh… So yeah, probably today I'm going to open the featured slugs and clean up PR. And. later this week will complete. So I've updated the project, moving on the post-mortem to the next week, since last week I had no time to. completed due to some circumstances. And, uh… That's mostly it. I still got some. PRs requiring my attention, like the crash reporting fix. So, yeah, no updates since last week. On it. So, basically, my concern about it is, uh, we may, uh, we may break the existing macOS collection while fixing iOS collection, so I need to… reproduce it, reproduce the crash and. this slowdown condition and normal conditions and see how it if it still works and nothing breaks and then I'll proceed with it."
},
{
"time": "16:05:49",
"speaker": "Dominik Kapusta",
"text": "Collection as in…"
},
{
"time": "16:05:52",
"speaker": "Alex Martemyanov",
"text": "The App Store reports."
},
{
"time": "16:05:53",
"speaker": "Dominik Kapusta",
"text": "Crosscode? Yeah, cross-collection, mhm."
},
{
"time": "16:05:57",
"speaker": "Alex Martemyanov",
"text": "Yeah, so matching against process identifier or by date."
},
{
"time": "16:06:02",
"speaker": "Dominik Kapusta",
"text": "Yeah. Yeah, yeah, yeah, yeah."
},
{
"time": "16:06:05",
"speaker": "Alex Martemyanov",
"text": "Mmm."
},
{
"time": "16:06:05",
"speaker": "Dominik Kapusta",
"text": "Oh. Right."
},
{
"time": "16:06:14",
"speaker": "Alex Martemyanov",
"text": "And… That is it, I guess, so… I'm. I'm not top performing for the last week. So just… Just completing the TP stuff."
},
{
"time": "16:06:35",
"speaker": "Dominik Kapusta",
"text": "Yeah, which is…"
},
{
"time": "16:06:36",
"speaker": "Alex Martemyanov",
"text": "And yeah, there was…"
},
{
"time": "16:06:38",
"speaker": "Dominik Kapusta",
"text": "Expected."
},
{
"time": "16:06:39",
"speaker": "Alex Martemyanov",
"text": "And there was the maintenance, the release maintenance. Luckily, it was growing weak."
},
{
"time": "16:06:46",
"speaker": "Dominik Kapusta",
"text": "Mm-hmm."
},
{
"time": "16:06:47",
"speaker": "Alex Martemyanov",
"text": "So nothing else ended from it."
},
{
"time": "16:06:48",
"speaker": "Dominik Kapusta",
"text": "I saw that, yeah, that's good, that's good. All right. Nice. That's fine. So you're going to wrap up your project. Will you run the postmortem this week or next week?"
},
{
"time": "16:07:05",
"speaker": "Alex Martemyanov",
"text": "Next week, next week."
},
{
"time": "16:07:06",
"speaker": "Dominik Kapusta",
"text": "Next week. Mm-hmm. Okay."
},
{
"time": "16:07:07",
"speaker": "Alex Martemyanov",
"text": "So this week, I'm, uh… Completing the cleanup. And the next week is time for the postmortem."
},
{
"time": "16:07:18",
"speaker": "Dominik Kapusta",
"text": "Sounds good. Alright, and then… Do you have… Do you know anything about… You're next. Top priority?"
},
{
"time": "16:07:32",
"speaker": "Alex Martemyanov",
"text": "Uh, nope."
},
{
"time": "16:07:35",
"speaker": "Dominik Kapusta",
"text": "Yeah, because there won't be any follow-up… there won't be any follow-up in OE for…"
},
{
"time": "16:07:35",
"speaker": "Alex Martemyanov",
"text": "I guess I should make one."
},
{
"time": "16:07:41",
"speaker": "Dominik Kapusta",
"text": "For this. I mean…"
},
{
"time": "16:07:44",
"speaker": "Alex Martemyanov",
"text": "Well, I guess not, at least for the moment."
},
{
"time": "16:07:45",
"speaker": "Dominik Kapusta",
"text": "Immediate follow up at least. Yeah, yeah, yeah."
},
{
"time": "16:07:49",
"speaker": "Alex Martemyanov",
"text": "And, uh… So if we are shifting the diagnostics into the JavaScript part, it's probably… due to some someone else to. Keep up with it."
},
{
"time": "16:08:05",
"speaker": "Dominik Kapusta",
"text": "No."
},
{
"time": "16:08:05",
"speaker": "Alex Martemyanov",
"text": "So maybe there will be a follow-up task to perform the data analysis next week."
},
{
"time": "16:08:12",
"speaker": "Dominik Kapusta",
"text": "Mm-hmm."
},
{
"time": "16:08:12",
"speaker": "Alex Martemyanov",
"text": "So just one, one cloud shot. To fetch the data, to perform the analysis, to see the trends, and if… Anything new is highlighted from it. So I could person. Wrap up up a date. And that's it. And it goes after."
},
{
"time": "16:08:29",
"speaker": "Dominik Kapusta",
"text": "Yeah, yeah."
},
{
"time": "16:08:31",
"speaker": "Alex Martemyanov",
"text": "Two weeks from now, I guess, when the release is out."
},
{
"time": "16:08:39",
"speaker": "Dominik Kapusta",
"text": "All right, fine, we'll need to learn what's next for you. When would you be opening? When would you be opening a top priority, Fred?"
},
{
"time": "16:08:47",
"speaker": "Alex Martemyanov",
"text": "Gosh. Probably now."
},
{
"time": "16:08:53",
"speaker": "Dominik Kapusta",
"text": "Yep."
},
{
"time": "16:08:55",
"speaker": "Alex Martemyanov",
"text": "It's really, it's a good time."
},
{
"time": "16:08:55",
"speaker": "Dominik Kapusta",
"text": "Okay. Okay, this is interesting then. There will be some work. shaping up in OB. If you're up for it, that may be interesting. I'm not sure how much native support they need, but… It is… considered a big problem these days, and that's the fact that all the JavaScript code injected into our websites causes performance issues. And side breakage. But… Yeah, well, both of them, essentially. And I'm, like I said, I'm not sure… If… A native engineer would be needed immediately. But if so, then… that… That would be very important. So this is very important. And… Let me see… So the assessment is this Thursday. And the projects… Okay. Let's see. Let's see. I see there is a… So this is the assessment task. And there is a subtask for the… staffing for the Apple team, so it sounds like they will need… they will need someone from the Apple team. This is not finalized yet, but… But, uh… Should probably be… should probably be… filled out today, I hope. I'll have the proposed critical path and Gantt chart ready for review on Tuesday. I'm proposing OB focus on performance this cycle, specifically the cost of the JS and config we inject into every frame."
},
{
"time": "16:11:21",
"speaker": "Alex Martemyanov",
"text": "Thank you."
},
{
"time": "16:11:21",
"speaker": "Dominik Kapusta",
"text": "They may at least need someone on the native side to, I don't know, implement some. measurements. I suppose. If this is something that… would be of interest to you, then that's great, because if not, I think you will still be mentioned there. What do you think?"
},
{
"time": "16:11:58",
"speaker": "Alex Martemyanov",
"text": "Visibility. What is this discussion? That you mentioned."
},
{
"time": "16:12:04",
"speaker": "Dominik Kapusta",
"text": "I was just reading the last comment in the thread."
},
{
"time": "16:12:07",
"speaker": "Alex Martemyanov",
"text": "Okay."
},
{
"time": "16:12:08",
"speaker": "Dominik Kapusta",
"text": "From Dave."
},
{
"time": "16:12:22",
"speaker": "Alex Martemyanov",
"text": "Mmm… Well, looking at my current TP, if it can be reproduced, it would be more of a… something that I would be interested in, if… And if it's something like we got some breakage to fix it. Not so exciting, I'd say."
},
{
"time": "16:12:52",
"speaker": "Dominik Kapusta",
"text": "Yeah, yeah, yeah, I can tell. I can tell. This, however, should be easier. On the functional side, on the technical side, it should be easier. at least from…"
},
{
"time": "16:13:10",
"speaker": "Alex Martemyanov",
"text": "Yeah, at least if there is exact cases reported on or some exact websites."
},
{
"time": "16:13:13",
"speaker": "Dominik Kapusta",
"text": "Yeah, I know. Uh, it may be… it may be about… it may be about just measuring things."
},
{
"time": "16:13:17",
"speaker": "Alex Martemyanov",
"text": "It should be easier."
},
{
"time": "16:13:22",
"speaker": "Dominik Kapusta",
"text": "So that front-end engineers can… Do changes, make changes. But I don't know. I don't know the details. Let's see. Trim the Apple Cs. Bundle and isolated world."
},
{
"time": "16:13:37",
"speaker": "Alex Martemyanov",
"text": "Okay, I'm fine to get mentioned on the thread, and so for the next TP, I'll probably open it on Thursday, I guess."
},
{
"time": "16:13:38",
"speaker": "Dominik Kapusta",
"text": "Interesting. Mmhm."
},
{
"time": "16:13:47",
"speaker": "Alex Martemyanov",
"text": "Makes sense."
},
{
"time": "16:13:49",
"speaker": "Dominik Kapusta",
"text": "Yeah, probably. I think it makes a lot of sense. I will be AFK for the rest of the week. Um, Bob…"
},
{
"time": "16:13:57",
"speaker": "Alex Martemyanov",
"text": "Mm-hmm."
},
{
"time": "16:13:59",
"speaker": "Dominik Kapusta",
"text": "It's not a problem, of course. Um… I will try to… Try to reach out to Jonathan to ask. If he knows anything about OB plans for a pool. Or if he knows anything about anything else. Any other objective that might need Apple people and I'll let you know. Do you want to talk about your performance review?"
},
{
"time": "16:14:31",
"speaker": "Alex Martemyanov",
"text": "Uh, yes, I guess so."
},
{
"time": "16:14:35",
"speaker": "Dominik Kapusta",
"text": "Why not? Right. All right. I linked it in the agenda, updated the link to the latest review. So, congrats on the promotion!"
},
{
"time": "16:14:52",
"speaker": "Alex Martemyanov",
"text": "Thanks."
},
{
"time": "16:14:52",
"speaker": "Dominik Kapusta",
"text": "Amazing, you made it."
},
{
"time": "16:14:53",
"speaker": "Alex Martemyanov",
"text": "I guess it was a lot of your efforts."
},
{
"time": "16:14:55",
"speaker": "Dominik Kapusta",
"text": "uh…"
},
{
"time": "16:15:00",
"speaker": "Alex Martemyanov",
"text": "Involved to make it go through."
},
{
"time": "16:15:06",
"speaker": "Dominik Kapusta",
"text": "Just a bit, really. It's, you know, this… this performance review time is overall great. I mean, I like it. I like… I like working on it, because it's, uh…"
},
{
"time": "16:15:17",
"speaker": "Alex Martemyanov",
"text": "Bye."
},
{
"time": "16:15:19",
"speaker": "Dominik Kapusta",
"text": "Um… okay. It's a great time. When you can appreciate people you work with and be able to do something for them. uh… make them… make their efforts visible and recognized. So, if that works, then that's great. This time it worked, so that's amazing. Uh, but… At the end of the day, it's just your work. put into the project. Uh, and, uh, and that got you… a bump in project management, which is amazing, which is… it's essentially hard facts, especially this year. If you look at the… if you look at the review… Format. It's… Like, dry facts. Nothing else, no pros, no…"
},
{
"time": "16:16:13",
"speaker": "Alex Martemyanov",
"text": "Mm-hmm."
},
{
"time": "16:16:15",
"speaker": "Dominik Kapusta",
"text": "Did a great job! everyone likes you, and so on and so on, like it used to be in the previous years. This time, it's, like, super, super concrete. With dates, with complexity assessments, with… The amount of direction needed."
},
{
"time": "16:16:32",
"speaker": "Alex Martemyanov",
"text": "Mmhm."
},
{
"time": "16:16:33",
"speaker": "Dominik Kapusta",
"text": "with clear impact stated in each bullet point. So it's just your work. distilled into a few sentences. So… Yeah, great job and keep doing that. I have to say… I have to say, I didn't ask for a promotion, because… I learned that these are the rules. The rules are, you have to have a star in all… in 4 out of 5 dimensions. So you would have to have a star in project management, and in… Team leadership. So I was like, okay, I would like to bump Alexa's project management. But I'm keeping the level at D, because… That's what the rules are, and the rules are in Asana. Um… These are the rules. Alex. There is a task about it. So I read through that task to be 100% sure that I'm doing the right thing, and I said. We have four stars in total, but not… in four dimensions, so it remains D. And everyone agreed with me. And then the calibration committee visited the review just on… it's actually last Friday afternoon. And they said, well, we decided to promote Alex based on that, because… Because we… that's what they thought would be the right thing, and also they are working on updating these rules here. to allow promotions in more, like, spiky profiles, when you have two stars somewhere, and no stars elsewhere. Uh, so…"
},
{
"time": "16:18:26",
"speaker": "Alex Martemyanov",
"text": "Mm-hmm. No, I see."
},
{
"time": "16:18:31",
"speaker": "Dominik Kapusta",
"text": "Yeah, that… that happened. Oh, actually…"
},
{
"time": "16:18:31",
"speaker": "Alex Martemyanov",
"text": "Thank you."
},
{
"time": "16:18:35",
"speaker": "Dominik Kapusta",
"text": "Oh, actually, there is a change in that. If you see… The latest edit in that task. there is a sentence added. In exceptional cases, someone may have spiky dimensions performing more than one level above the next level. these can balance out dimensions that are not yet at the next level and will be reviewed case by case. So, in your case, that warranted a promotion, so that's great. Um…"
},
{
"time": "16:19:00",
"speaker": "Alex Martemyanov",
"text": "I see."
},
{
"time": "16:19:01",
"speaker": "Dominik Kapusta",
"text": "All right, so we have this. Um, and… Well, you proved that you're… operating on the D-star level when it comes to project management. Um, and uh… Where is your review again? Yes. So we have this, we… I didn't have the… evidence to… ask… bump Team Leadership to the star, but this is something we should… focus on… In the coming year."
},
{
"time": "16:19:48",
"speaker": "Alex Martemyanov",
"text": "Okay."
},
{
"time": "16:19:48",
"speaker": "Dominik Kapusta",
"text": "And it will be… Tricky. Because you're not a project advisor. And it will be… Even trickier, because you may… In your current… Work. setting. It may be tricky for you to become a project advisor, but… Let's do whatever we can to try and push that. So, in short, to become a project advisor, you should probably be able to stay in a single objective for some time. On a positive side, we have 12 months. Or six months, let's say, until you're able to operate. to work as a project advisor and make impact, right? And also, I wanted to… Clarify that it's not required. It's not required that you grow to D-star in team leadership by next cycle. It would just be great, it would be the right thing. It's just what's missing in the distal level, so… So we should focus on that specifically. Uh, team leadership… is… Essentially about accelerating the work that is being done. And it's a quote from the Professional Levels PDF. Organizational leadership is accelerating how we work. overall, like, overall, and team leadership is accelerating the work that is currently being done. So, organizational leadership is figuring out new processes, or improving existing processes, and team leadership is finding people that. Struggle at a given time and helping them move forward."
},
{
"time": "16:21:49",
"speaker": "Alex Martemyanov",
"text": "Mmhm."
},
{
"time": "16:21:51",
"speaker": "Dominik Kapusta",
"text": "And, uh… the project advisor role fits here very, very well. So it's very easy, if you're a project advisor, if you're advising projects, then you can just help move projects forward. If you're not a project advisor, you can still do it. You can still do it, but it requires you to proactively go. Go through projects and see if there's something you can help somewhere. And I imagine… It can be very hard if… You're just starting out on that? So… and I'm also… You know, we have plenty of time. But, uh, it would be great to have. Some evidence of you doing things like that. In a year from now."
},
{
"time": "16:22:44",
"speaker": "Alex Martemyanov",
"text": "Mm-hmm."
},
{
"time": "16:22:47",
"speaker": "Dominik Kapusta",
"text": "Let's see, I'll try to… I'll be trying to… let you know about opportunities to do that."
},
{
"time": "16:22:58",
"speaker": "Alex Martemyanov",
"text": "Okay, so…"
},
{
"time": "16:22:58",
"speaker": "Dominik Kapusta",
"text": "You could also, if you have… Yeah, you could also, like… Look at some projects here and there. OL seems… straightforward, the most straightforward here. However, OL these days is about web extensions, mostly. And we seem to be covered there, but if there's anything… coming up in OL that I think would benefit from your help. Then the easiest is to just add yourself to the project and keep following whatever is happening there. Look at the subtasks. I don't know, help? help the DRI essentially move forward, um… And let's see how it goes. We'll… We will revisit growth areas, and also… Try to formulate something that… Not really, like, describes… What needs to happen? So that's team leadership, and that should be the main focus for next cycle. Uh, apart from that, just keep up your project management, and also getting to this start in project management, showing that you can… run projects smoothly and keep stakeholders up to date and essentially. yeah, do a good job as a… as a project DRI is… is something that will help you become a project advisor, I believe, uh, I'm sure. But the process… the process is that the ODRI selects new project advisors, and this is the main blocker now."
},
{
"time": "16:24:36",
"speaker": "Alex Martemyanov",
"text": "Okay, yeah."
},
{
"time": "16:24:44",
"speaker": "Dominik Kapusta",
"text": "So, just to end my speech… Even if you're… bored with… Uh, analyzing site breakage, or various types of breakage. Maybe if it's something that you have actually… Uh, you have a lot of experience in… maybe it would be… easier, helpful to… To stay in this area. If that's possible, if… You know, if staffing allows for that. Uh, and gain a little bit more depth. which would allow you to grow others, perhaps advise projects at some point. But let's see. For now… Take it easy. Take it easy. It's not that you have to start smashing it first week after the review. So yeah, wrap up your project. Let's see what comes next. If the next thing for you is OB, then I think it will be good. If not, we'll rearrange things and see how can we grow in."
},
{
"time": "16:25:55",
"speaker": "Alex Martemyanov",
"text": "Mm-hmm."
},
{
"time": "16:26:03",
"speaker": "Dominik Kapusta",
"text": "In a different setting, but yeah. That's what I wanted to say overall. What do you think about it?"
},
{
"time": "16:26:10",
"speaker": "Alex Martemyanov",
"text": "Yeah, sounds wise, uh… I guess so. There is. some… opportunities to be looked for in the tech designs and. Some other projects that are ongoing, related to my expertise, and uh…"
},
{
"time": "16:26:28",
"speaker": "Dominik Kapusta",
"text": "Yeah, totally."
},
{
"time": "16:26:31",
"speaker": "Alex Martemyanov",
"text": "And… Yeah, let's look for… for the OB, uh… Next project."
},
{
"time": "16:26:39",
"speaker": "Dominik Kapusta",
"text": "Mmhm."
},
{
"time": "16:26:40",
"speaker": "Alex Martemyanov",
"text": "Maybe there's something that could be done."
},
{
"time": "16:26:46",
"speaker": "Dominik Kapusta",
"text": "Cool. Cool, cool, cool. That's, that sounds good. All right, and with that, like I said, for now, let's just see what happens. Let's see where you land next next week. Mmhm. And I think that's all I had. Can I help you with anything? Is there anything you wanted to… Raise. talk about."
},
{
"time": "16:27:22",
"speaker": "Alex Martemyanov",
"text": "Okay. I… I don't know. Not really, not anything. In this tech, um…"
},
{
"time": "16:27:48",
"speaker": "Dominik Kapusta",
"text": "That's fine. That's fine."
},
{
"time": "16:27:52",
"speaker": "Alex Martemyanov",
"text": "So, yeah, there was, uh, some work, like, kinda planned for, uh, the UI test runners and, uh, all this stuff, but, uh… I'm… At the moment, I'm not doing anything related to it, just completing the TPM."
},
{
"time": "16:28:09",
"speaker": "Dominik Kapusta",
"text": "Mm-hmm."
},
{
"time": "16:28:10",
"speaker": "Alex Martemyanov",
"text": "Um, the open… They opened, they already closed pull requests that I need to complete and. That's… that's all that's on my plate. At the moment, so I'm planning to move forward with this."
},
{
"time": "16:28:28",
"speaker": "Dominik Kapusta",
"text": "Mmhm."
},
{
"time": "16:28:28",
"speaker": "Alex Martemyanov",
"text": "organizational leadership stuff."
},
{
"time": "16:28:32",
"speaker": "Dominik Kapusta",
"text": "Okay."
},
{
"time": "16:28:32",
"speaker": "Alex Martemyanov",
"text": "But not… not right now."
},
{
"time": "16:28:37",
"speaker": "Dominik Kapusta",
"text": "Sure, sure. I mean, that's fine. Totally fine to… Like I said, to take it easy for now. Um, alright, uh…"
},
{
"time": "16:28:52",
"speaker": "Alex Martemyanov",
"text": "I mean, for sure, I… I will need to, uh, sync up with you, and… On that area, when I get my hands on it."
},
{
"time": "16:29:00",
"speaker": "Dominik Kapusta",
"text": "Mm-hmm. Yeah, yeah, we can do it next."
},
{
"time": "16:29:03",
"speaker": "Alex Martemyanov",
"text": "But yeah, just not ready."
},
{
"time": "16:29:05",
"speaker": "Dominik Kapusta",
"text": "next week, um… Later. Should be fine. All right, all right."
},
{
"time": "16:29:15",
"speaker": "Alex Martemyanov",
"text": "Thank you."
},
{
"time": "16:29:20",
"speaker": "Dominik Kapusta",
"text": "Good. So if nothing else, we can wrap it up here. I don't have any other announcements, any other important info. I'm also… deep down in my project, trying to open a ship review today, so let's see. And then I'm taking 3 days off, finally. So I'll be back next week."
},
{
"time": "16:29:43",
"speaker": "Alex Martemyanov",
"text": "Okay."
},
{
"time": "16:29:43",
"speaker": "Dominik Kapusta",
"text": "But we'll see each other weekly. If I know anything from Jonathan, I'll let you know. Because I'm curious myself. So let's see. Good. If that's it, we can wrap it up here. Congrats again. You're doing a great job. It's a pleasure to be working with you, and yeah, we'll keep smashing it, hopefully, starting next week."
},
{
"time": "16:30:05",
"speaker": "Alex Martemyanov",
"text": "Thanks."
},
{
"time": "16:30:12",
"speaker": "Dominik Kapusta",
"text": "Awesome."
},
{
"time": "16:30:13",
"speaker": "Alex Martemyanov",
"text": "Okay."
},
{
"time": "16:30:14",
"speaker": "Dominik Kapusta",
"text": "Thank you. Thanks and have a great rest of the day. See ya."
}
]
}
@@ -14,3 +14,130 @@ source: "https://us01mynotes.zoom.us/p5FdcX9WT-afBpWjRgaW0w"
- **16:01:44 · Dominik Kapusta:** Is it going.
- **16:01:46 · Alex Martemyanov:** And then… I'm pretty good. Thank you. After you… after last week's, uh… I was… So all this RKAFK, so I'm… Wrapping up the top priority project. Mostly, mostly it finally merged the link tooltip thing from the Hug days.
- **16:02:10 · Dominik Kapusta:** Jest.
- **16:02:13 · Alex Martemyanov:** And basically no progress on other stuff yet.
- **16:02:19 · Alex Martemyanov:** I need to finish the priority cleanup. So basically last week we've discussed it during the OE that we should. Basically wrap up the project with the. No visible outcomes. There was a slight decline in the numbers and several genuine bugs fixed, but they were not representing the main feature breakage.
- **16:02:45 · Dominik Kapusta:** Nie. Drop, Ios.
- **16:02:50 · Alex Martemyanov:** I know.
- **16:02:57 · Dominik Kapusta:** Łazać.
- **16:03:00 · Alex Martemyanov:** There was some decline, maybe a third. So there was a decline in some part. of the numbers, but there is still around 40%. of the event sequence of the received events that so basically most of it is. The data we received for most of the breakage pixels represents just regular normal flow and nothing obviously.
- **16:03:34 · Dominik Kapusta:** Bardzo.
- **16:03:34 · Alex Martemyanov:** pointing to anything. So the discussion during the OE was to basically clean up the inefficient fixes that we've got. The feature flags for the features that are removed and some. Um, like the pinned tabs management fix, uh, that are, um, staying without the feature flag. And, uh… Yeah, clean up the data that's being collected, because we've collected enough of it, and, uh, implement some permanent data collection that stays there for, uh. for diagnostics for the future possible breakages. And try to add some in. inside of the extension, some metrics inside of the extension to, um, to track the loading progress, basically. Where we… where do we stop, and where the messaging.
- **16:04:23 · Dominik Kapusta:** Ok.
- **16:04:31 · Alex Martemyanov:** Stops. and try to reason about it from from that perspective.
- **16:04:37 · Dominik Kapusta:** Misji.
- **16:04:39 · Alex Martemyanov:** So, uh… So yeah, probably today I'm going to open the featured slugs and clean up PR. And. later this week will complete. So I've updated the project, moving on the post-mortem to the next week, since last week I had no time to. completed due to some circumstances. And, uh… That's mostly it. I still got some. PRs requiring my attention, like the crash reporting fix. So, yeah, no updates since last week. On it. So, basically, my concern about it is, uh, we may, uh, we may break the existing macOS collection while fixing iOS collection, so I need to… reproduce it, reproduce the crash and. this slowdown condition and normal conditions and see how it if it still works and nothing breaks and then I'll proceed with it.
- **16:05:49 · Dominik Kapusta:** Collection as in…
- **16:05:52 · Alex Martemyanov:** The App Store reports.
- **16:05:53 · Dominik Kapusta:** Crosscode? Yeah, cross-collection, mhm.
- **16:05:57 · Alex Martemyanov:** Yeah, so matching against process identifier or by date.
- **16:06:02 · Dominik Kapusta:** Yeah. Yeah, yeah, yeah, yeah.
- **16:06:05 · Alex Martemyanov:** Mmm.
- **16:06:05 · Dominik Kapusta:** Oh. Right.
- **16:06:14 · Alex Martemyanov:** And… That is it, I guess, so… I'm. I'm not top performing for the last week. So just… Just completing the TP stuff.
- **16:06:35 · Dominik Kapusta:** Yeah, which is…
- **16:06:36 · Alex Martemyanov:** And yeah, there was…
- **16:06:38 · Dominik Kapusta:** Expected.
- **16:06:39 · Alex Martemyanov:** And there was the maintenance, the release maintenance. Luckily, it was growing weak.
- **16:06:46 · Dominik Kapusta:** Mm-hmm.
- **16:06:47 · Alex Martemyanov:** So nothing else ended from it.
- **16:06:48 · Dominik Kapusta:** I saw that, yeah, that's good, that's good. All right. Nice. That's fine. So you're going to wrap up your project. Will you run the postmortem this week or next week?
- **16:07:05 · Alex Martemyanov:** Next week, next week.
- **16:07:06 · Dominik Kapusta:** Next week. Mm-hmm. Okay.
- **16:07:07 · Alex Martemyanov:** So this week, I'm, uh… Completing the cleanup. And the next week is time for the postmortem.
- **16:07:18 · Dominik Kapusta:** Sounds good. Alright, and then… Do you have… Do you know anything about… You're next. Top priority?
- **16:07:32 · Alex Martemyanov:** Uh, nope.
- **16:07:35 · Dominik Kapusta:** Yeah, because there won't be any follow-up… there won't be any follow-up in OE for…
- **16:07:35 · Alex Martemyanov:** I guess I should make one.
- **16:07:41 · Dominik Kapusta:** For this. I mean…
- **16:07:44 · Alex Martemyanov:** Well, I guess not, at least for the moment.
- **16:07:45 · Dominik Kapusta:** Immediate follow up at least. Yeah, yeah, yeah.
- **16:07:49 · Alex Martemyanov:** And, uh… So if we are shifting the diagnostics into the JavaScript part, it's probably… due to some someone else to. Keep up with it.
- **16:08:05 · Dominik Kapusta:** No.
- **16:08:05 · Alex Martemyanov:** So maybe there will be a follow-up task to perform the data analysis next week.
- **16:08:12 · Dominik Kapusta:** Mm-hmm.
- **16:08:12 · Alex Martemyanov:** So just one, one cloud shot. To fetch the data, to perform the analysis, to see the trends, and if… Anything new is highlighted from it. So I could person. Wrap up up a date. And that's it. And it goes after.
- **16:08:29 · Dominik Kapusta:** Yeah, yeah.
- **16:08:31 · Alex Martemyanov:** Two weeks from now, I guess, when the release is out.
- **16:08:39 · Dominik Kapusta:** All right, fine, we'll need to learn what's next for you. When would you be opening? When would you be opening a top priority, Fred?
- **16:08:47 · Alex Martemyanov:** Gosh. Probably now.
- **16:08:53 · Dominik Kapusta:** Yep.
- **16:08:55 · Alex Martemyanov:** It's really, it's a good time.
- **16:08:55 · Dominik Kapusta:** Okay. Okay, this is interesting then. There will be some work. shaping up in OB. If you're up for it, that may be interesting. I'm not sure how much native support they need, but… It is… considered a big problem these days, and that's the fact that all the JavaScript code injected into our websites causes performance issues. And side breakage. But… Yeah, well, both of them, essentially. And I'm, like I said, I'm not sure… If… A native engineer would be needed immediately. But if so, then… that… That would be very important. So this is very important. And… Let me see… So the assessment is this Thursday. And the projects… Okay. Let's see. Let's see. I see there is a… So this is the assessment task. And there is a subtask for the… staffing for the Apple team, so it sounds like they will need… they will need someone from the Apple team. This is not finalized yet, but… But, uh… Should probably be… should probably be… filled out today, I hope. I'll have the proposed critical path and Gantt chart ready for review on Tuesday. I'm proposing OB focus on performance this cycle, specifically the cost of the JS and config we inject into every frame.
- **16:11:21 · Alex Martemyanov:** Thank you.
- **16:11:21 · Dominik Kapusta:** They may at least need someone on the native side to, I don't know, implement some. measurements. I suppose. If this is something that… would be of interest to you, then that's great, because if not, I think you will still be mentioned there. What do you think?
- **16:11:58 · Alex Martemyanov:** Visibility. What is this discussion? That you mentioned.
- **16:12:04 · Dominik Kapusta:** I was just reading the last comment in the thread.
- **16:12:07 · Alex Martemyanov:** Okay.
- **16:12:08 · Dominik Kapusta:** From Dave.
- **16:12:22 · Alex Martemyanov:** Mmm… Well, looking at my current TP, if it can be reproduced, it would be more of a… something that I would be interested in, if… And if it's something like we got some breakage to fix it. Not so exciting, I'd say.
- **16:12:52 · Dominik Kapusta:** Yeah, yeah, yeah, I can tell. I can tell. This, however, should be easier. On the functional side, on the technical side, it should be easier. at least from…
- **16:13:10 · Alex Martemyanov:** Yeah, at least if there is exact cases reported on or some exact websites.
- **16:13:13 · Dominik Kapusta:** Yeah, I know. Uh, it may be… it may be about… it may be about just measuring things.
- **16:13:17 · Alex Martemyanov:** It should be easier.
- **16:13:22 · Dominik Kapusta:** So that front-end engineers can… Do changes, make changes. But I don't know. I don't know the details. Let's see. Trim the Apple Cs. Bundle and isolated world.
- **16:13:37 · Alex Martemyanov:** Okay, I'm fine to get mentioned on the thread, and so for the next TP, I'll probably open it on Thursday, I guess.
- **16:13:38 · Dominik Kapusta:** Interesting. Mmhm.
- **16:13:47 · Alex Martemyanov:** Makes sense.
- **16:13:49 · Dominik Kapusta:** Yeah, probably. I think it makes a lot of sense. I will be AFK for the rest of the week. Um, Bob…
- **16:13:57 · Alex Martemyanov:** Mm-hmm.
- **16:13:59 · Dominik Kapusta:** It's not a problem, of course. Um… I will try to… Try to reach out to Jonathan to ask. If he knows anything about OB plans for a pool. Or if he knows anything about anything else. Any other objective that might need Apple people and I'll let you know. Do you want to talk about your performance review?
- **16:14:31 · Alex Martemyanov:** Uh, yes, I guess so.
- **16:14:35 · Dominik Kapusta:** Why not? Right. All right. I linked it in the agenda, updated the link to the latest review. So, congrats on the promotion!
- **16:14:52 · Alex Martemyanov:** Thanks.
- **16:14:52 · Dominik Kapusta:** Amazing, you made it.
- **16:14:53 · Alex Martemyanov:** I guess it was a lot of your efforts.
- **16:14:55 · Dominik Kapusta:** uh…
- **16:15:00 · Alex Martemyanov:** Involved to make it go through.
- **16:15:06 · Dominik Kapusta:** Just a bit, really. It's, you know, this… this performance review time is overall great. I mean, I like it. I like… I like working on it, because it's, uh…
- **16:15:17 · Alex Martemyanov:** Bye.
- **16:15:19 · Dominik Kapusta:** Um… okay. It's a great time. When you can appreciate people you work with and be able to do something for them. uh… make them… make their efforts visible and recognized. So, if that works, then that's great. This time it worked, so that's amazing. Uh, but… At the end of the day, it's just your work. put into the project. Uh, and, uh, and that got you… a bump in project management, which is amazing, which is… it's essentially hard facts, especially this year. If you look at the… if you look at the review… Format. It's… Like, dry facts. Nothing else, no pros, no…
- **16:16:13 · Alex Martemyanov:** Mm-hmm.
- **16:16:15 · Dominik Kapusta:** Did a great job! everyone likes you, and so on and so on, like it used to be in the previous years. This time, it's, like, super, super concrete. With dates, with complexity assessments, with… The amount of direction needed.
- **16:16:32 · Alex Martemyanov:** Mmhm.
- **16:16:33 · Dominik Kapusta:** with clear impact stated in each bullet point. So it's just your work. distilled into a few sentences. So… Yeah, great job and keep doing that. I have to say… I have to say, I didn't ask for a promotion, because… I learned that these are the rules. The rules are, you have to have a star in all… in 4 out of 5 dimensions. So you would have to have a star in project management, and in… Team leadership. So I was like, okay, I would like to bump Alexa's project management. But I'm keeping the level at D, because… That's what the rules are, and the rules are in Asana. Um… These are the rules. Alex. There is a task about it. So I read through that task to be 100% sure that I'm doing the right thing, and I said. We have four stars in total, but not… in four dimensions, so it remains D. And everyone agreed with me. And then the calibration committee visited the review just on… it's actually last Friday afternoon. And they said, well, we decided to promote Alex based on that, because… Because we… that's what they thought would be the right thing, and also they are working on updating these rules here. to allow promotions in more, like, spiky profiles, when you have two stars somewhere, and no stars elsewhere. Uh, so…
- **16:18:26 · Alex Martemyanov:** Mm-hmm. No, I see.
- **16:18:31 · Dominik Kapusta:** Yeah, that… that happened. Oh, actually…
- **16:18:31 · Alex Martemyanov:** Thank you.
- **16:18:35 · Dominik Kapusta:** Oh, actually, there is a change in that. If you see… The latest edit in that task. there is a sentence added. In exceptional cases, someone may have spiky dimensions performing more than one level above the next level. these can balance out dimensions that are not yet at the next level and will be reviewed case by case. So, in your case, that warranted a promotion, so that's great. Um…
- **16:19:00 · Alex Martemyanov:** I see.
- **16:19:01 · Dominik Kapusta:** All right, so we have this. Um, and… Well, you proved that you're… operating on the D-star level when it comes to project management. Um, and uh… Where is your review again? Yes. So we have this, we… I didn't have the… evidence to… ask… bump Team Leadership to the star, but this is something we should… focus on… In the coming year.
- **16:19:48 · Alex Martemyanov:** Okay.
- **16:19:48 · Dominik Kapusta:** And it will be… Tricky. Because you're not a project advisor. And it will be… Even trickier, because you may… In your current… Work. setting. It may be tricky for you to become a project advisor, but… Let's do whatever we can to try and push that. So, in short, to become a project advisor, you should probably be able to stay in a single objective for some time. On a positive side, we have 12 months. Or six months, let's say, until you're able to operate. to work as a project advisor and make impact, right? And also, I wanted to… Clarify that it's not required. It's not required that you grow to D-star in team leadership by next cycle. It would just be great, it would be the right thing. It's just what's missing in the distal level, so… So we should focus on that specifically. Uh, team leadership… is… Essentially about accelerating the work that is being done. And it's a quote from the Professional Levels PDF. Organizational leadership is accelerating how we work. overall, like, overall, and team leadership is accelerating the work that is currently being done. So, organizational leadership is figuring out new processes, or improving existing processes, and team leadership is finding people that. Struggle at a given time and helping them move forward.
- **16:21:49 · Alex Martemyanov:** Mmhm.
- **16:21:51 · Dominik Kapusta:** And, uh… the project advisor role fits here very, very well. So it's very easy, if you're a project advisor, if you're advising projects, then you can just help move projects forward. If you're not a project advisor, you can still do it. You can still do it, but it requires you to proactively go. Go through projects and see if there's something you can help somewhere. And I imagine… It can be very hard if… You're just starting out on that? So… and I'm also… You know, we have plenty of time. But, uh, it would be great to have. Some evidence of you doing things like that. In a year from now.
- **16:22:44 · Alex Martemyanov:** Mm-hmm.
- **16:22:47 · Dominik Kapusta:** Let's see, I'll try to… I'll be trying to… let you know about opportunities to do that.
- **16:22:58 · Alex Martemyanov:** Okay, so…
- **16:22:58 · Dominik Kapusta:** You could also, if you have… Yeah, you could also, like… Look at some projects here and there. OL seems… straightforward, the most straightforward here. However, OL these days is about web extensions, mostly. And we seem to be covered there, but if there's anything… coming up in OL that I think would benefit from your help. Then the easiest is to just add yourself to the project and keep following whatever is happening there. Look at the subtasks. I don't know, help? help the DRI essentially move forward, um… And let's see how it goes. We'll… We will revisit growth areas, and also… Try to formulate something that… Not really, like, describes… What needs to happen? So that's team leadership, and that should be the main focus for next cycle. Uh, apart from that, just keep up your project management, and also getting to this start in project management, showing that you can… run projects smoothly and keep stakeholders up to date and essentially. yeah, do a good job as a… as a project DRI is… is something that will help you become a project advisor, I believe, uh, I'm sure. But the process… the process is that the ODRI selects new project advisors, and this is the main blocker now.
- **16:24:36 · Alex Martemyanov:** Okay, yeah.
- **16:24:44 · Dominik Kapusta:** So, just to end my speech… Even if you're… bored with… Uh, analyzing site breakage, or various types of breakage. Maybe if it's something that you have actually… Uh, you have a lot of experience in… maybe it would be… easier, helpful to… To stay in this area. If that's possible, if… You know, if staffing allows for that. Uh, and gain a little bit more depth. which would allow you to grow others, perhaps advise projects at some point. But let's see. For now… Take it easy. Take it easy. It's not that you have to start smashing it first week after the review. So yeah, wrap up your project. Let's see what comes next. If the next thing for you is OB, then I think it will be good. If not, we'll rearrange things and see how can we grow in.
- **16:25:55 · Alex Martemyanov:** Mm-hmm.
- **16:26:03 · Dominik Kapusta:** In a different setting, but yeah. That's what I wanted to say overall. What do you think about it?
- **16:26:10 · Alex Martemyanov:** Yeah, sounds wise, uh… I guess so. There is. some… opportunities to be looked for in the tech designs and. Some other projects that are ongoing, related to my expertise, and uh…
- **16:26:28 · Dominik Kapusta:** Yeah, totally.
- **16:26:31 · Alex Martemyanov:** And… Yeah, let's look for… for the OB, uh… Next project.
- **16:26:39 · Dominik Kapusta:** Mmhm.
- **16:26:40 · Alex Martemyanov:** Maybe there's something that could be done.
- **16:26:46 · Dominik Kapusta:** Cool. Cool, cool, cool. That's, that sounds good. All right, and with that, like I said, for now, let's just see what happens. Let's see where you land next next week. Mmhm. And I think that's all I had. Can I help you with anything? Is there anything you wanted to… Raise. talk about.
- **16:27:22 · Alex Martemyanov:** Okay. I… I don't know. Not really, not anything. In this tech, um…
- **16:27:48 · Dominik Kapusta:** That's fine. That's fine.
- **16:27:52 · Alex Martemyanov:** So, yeah, there was, uh, some work, like, kinda planned for, uh, the UI test runners and, uh, all this stuff, but, uh… I'm… At the moment, I'm not doing anything related to it, just completing the TPM.
- **16:28:09 · Dominik Kapusta:** Mm-hmm.
- **16:28:10 · Alex Martemyanov:** Um, the open… They opened, they already closed pull requests that I need to complete and. That's… that's all that's on my plate. At the moment, so I'm planning to move forward with this.
- **16:28:28 · Dominik Kapusta:** Mmhm.
- **16:28:28 · Alex Martemyanov:** organizational leadership stuff.
- **16:28:32 · Dominik Kapusta:** Okay.
- **16:28:32 · Alex Martemyanov:** But not… not right now.
- **16:28:37 · Dominik Kapusta:** Sure, sure. I mean, that's fine. Totally fine to… Like I said, to take it easy for now. Um, alright, uh…
- **16:28:52 · Alex Martemyanov:** I mean, for sure, I… I will need to, uh, sync up with you, and… On that area, when I get my hands on it.
- **16:29:00 · Dominik Kapusta:** Mm-hmm. Yeah, yeah, we can do it next.
- **16:29:03 · Alex Martemyanov:** But yeah, just not ready.
- **16:29:05 · Dominik Kapusta:** next week, um… Later. Should be fine. All right, all right.
- **16:29:15 · Alex Martemyanov:** Thank you.
- **16:29:20 · Dominik Kapusta:** Good. So if nothing else, we can wrap it up here. I don't have any other announcements, any other important info. I'm also… deep down in my project, trying to open a ship review today, so let's see. And then I'm taking 3 days off, finally. So I'll be back next week.
- **16:29:43 · Alex Martemyanov:** Okay.
- **16:29:43 · Dominik Kapusta:** But we'll see each other weekly. If I know anything from Jonathan, I'll let you know. Because I'm curious myself. So let's see. Good. If that's it, we can wrap it up here. Congrats again. You're doing a great job. It's a pleasure to be working with you, and yeah, we'll keep smashing it, hopefully, starting next week.
- **16:30:05 · Alex Martemyanov:** Thanks.
- **16:30:12 · Dominik Kapusta:** Awesome.
- **16:30:13 · Alex Martemyanov:** Okay.
- **16:30:14 · Dominik Kapusta:** Thank you. Thanks and have a great rest of the day. See ya.
@@ -19,6 +19,566 @@
"time": "15:55:00",
"speaker": "Max Tsoy",
"text": "Yes, but still, like, I would expect… it's… it's not… it's all breakage reports, right? So… Yes, it is filtered by people who report something, but it's still a signal and it's not related to what kind of breakage report it is."
},
{
"time": "15:55:21",
"speaker": "Michal Smaga",
"text": "And you filter by the macOS version as well, right?"
},
{
"time": "15:55:24",
"speaker": "Max Tsoy",
"text": "Um… I think so. Yes, it's a stacked thing. It's in the same… in the same… Dashboards. Uh, this one. Right, so, uh, it's hidden under CPM diagnostics. Role."
},
{
"time": "15:55:51",
"speaker": "Alex Martemyanov",
"text": "Yeah, and just answered in your removed breakage diagnostic thread. And yeah, I basically asked Claude to check the breakages and related reports."
},
{
"time": "15:55:51",
"speaker": "Max Tsoy",
"text": "That's what I… Mm-hmm. Oh. Okay. Good."
},
{
"time": "15:56:06",
"speaker": "Alex Martemyanov",
"text": "So, it highlighted that, uh… The baseline is basically the same with the waiting signal, but there is a new report categories. Uh, that may be affecting, uh. that weighting metric and."
},
{
"time": "15:56:24",
"speaker": "Max Tsoy",
"text": "Okay, so…"
},
{
"time": "15:56:27",
"speaker": "Alex Martemyanov",
"text": "So…"
},
{
"time": "15:56:27",
"speaker": "Max Tsoy",
"text": "So we… we got more reports?"
},
{
"time": "15:56:29",
"speaker": "Alex Martemyanov",
"text": "if there is a report flow on the error page, uh, which basically has no, uh, CPM running on it, and we've been affected, uh, with it with the, uh, CPM breakage pixels as well, uh, because, uh, they're not run on determination and there are pages on macOS."
},
{
"time": "15:56:37",
"speaker": "Max Tsoy",
"text": "Okay."
},
{
"time": "15:56:47",
"speaker": "Alex Martemyanov",
"text": "And if we exclude the report flow for the error page, it stays steady basically. And answering your treatment effect question, yeah, basically it was not…"
},
{
"time": "15:56:53",
"speaker": "Max Tsoy",
"text": "Okay, okay."
},
{
"time": "15:57:02",
"speaker": "Alex Martemyanov",
"text": "lowering much, not affecting much the overall breakage signal magic."
},
{
"time": "15:57:08",
"speaker": "Max Tsoy",
"text": "Mm-hmm. Okay, so…"
},
{
"time": "15:57:10",
"speaker": "Alex Martemyanov",
"text": "Because the main signal still stays in the seemingly normal flow as the metrics shown."
},
{
"time": "15:57:17",
"speaker": "Max Tsoy",
"text": "Right. Okay, so basically, the increase is probably driven by something else, but there was no decrease either. Um…"
},
{
"time": "15:57:30",
"speaker": "Alex Martemyanov",
"text": "There was a… Probably less than a percent decrease and there's still fluctuation depending on a week measured, so we can tell for sure."
},
{
"time": "15:57:34",
"speaker": "Max Tsoy",
"text": "Yeah, yeah. Okay. Got it. Okay, so, uh, uh, alright, now, uh, I'm looking at your tech design, and my main question there is, do we… Do we not already have this? Because… so the tech design is basically proposing to send more, like, status reports from the extension."
},
{
"time": "15:58:04",
"speaker": "Alex Martemyanov",
"text": "Yep."
},
{
"time": "15:58:05",
"speaker": "Max Tsoy",
"text": "But we already have one signal, which is exactly the same thing, actually. So, if you look at this old… that design. that introduced that… other signal. So, the CPM dashboard state… Uh, specifically. Um… So basically, immediately after navigation, we set the CPM dashboard state to waiting in native code. And then… The first thing that… The extension does. After it gets… after it injects the content script. it's… initializes, and then… Almost immediately sends back the refresh CPM dashboard state. So… The problem is that. In the error state, this message is not received. So… Like, what you're proposing there… I think you will just not receive. Most of these messages. And that's the whole point of problem, right? So, like. Um… I don't think it's a problem of the granularity. It's basically that in this error state, we don't get. Any response?"
},
{
"time": "15:59:37",
"speaker": "Alex Martemyanov",
"text": "Mmhm."
},
{
"time": "15:59:37",
"speaker": "Max Tsoy",
"text": "So, it kind of looks like the messaging is broken. And that's the whole reason, right? Like, we don't know what's happening. Is the extension stuck? Is it just the messaging that doesn't get through, or is it the… You know, the whole… Thing doesn't work."
},
{
"time": "15:59:55",
"speaker": "Alex Martemyanov",
"text": "I think it was mainly based on the, um… In the state that may be pending before we send the… The repair is unmarked state. Like, uh, for example, the config, uh, the config update state."
},
{
"time": "16:00:09",
"speaker": "Max Tsoy",
"text": "Right."
},
{
"time": "16:00:14",
"speaker": "Alex Martemyanov",
"text": "or. And. What else was there? Let me just open it."
},
{
"time": "16:00:21",
"speaker": "Max Tsoy",
"text": "Right, so basically you're suggesting that within that initialization, maybe it gets stuck. In between these first… so, what we could do…"
},
{
"time": "16:00:28",
"speaker": "Alex Martemyanov",
"text": "Yes, yes."
},
{
"time": "16:00:32",
"speaker": "Max Tsoy",
"text": "We could just send one message. Immediately, like, on… On wake up. And I think we actually already do. Uh, don't we, like, there is some… I don't know, um, I don't remember exactly, but the… I think there was… so… What I'm saying is that I don't know if the, uh, all these other stages are… Um… useful, because I think it… the problem is that we… I suspect we won't get the first message, even."
},
{
"time": "16:01:07",
"speaker": "Alex Martemyanov",
"text": "Maybe, but… Probably we don't know."
},
{
"time": "16:01:15",
"speaker": "Max Tsoy",
"text": "So we have this… the first…"
},
{
"time": "16:01:15",
"speaker": "Alex Martemyanov",
"text": "So here we kind of add no wait, so there is no await, it's kind of asynchronously as soon as we…"
},
{
"time": "16:01:24",
"speaker": "Max Tsoy",
"text": "Mm-hmm."
},
{
"time": "16:01:26",
"speaker": "Alex Martemyanov",
"text": "Wake of the script wakes. Unless so first. Yeah, so there is the stage for, like, no, nothing was received, then, uh… H script in it. There may be possible hang on the config read, and if it hangs for whatever reason, it won't get through and won't get to the dashboard update state. Uh, so the suggestion is, like, sending, uh… And… log messages to… To set the list. Received state. that we could report in the main CPM breakage pixel."
},
{
"time": "16:02:15",
"speaker": "Max Tsoy",
"text": "Yeah, um…"
},
{
"time": "16:02:16",
"speaker": "Alex Martemyanov",
"text": "I see."
},
{
"time": "16:02:18",
"speaker": "Max Tsoy",
"text": "Well, then I would… I would suggest just to trim it down to, like, the very first. signal, because… Um, I think for the rest, we actually… we have some… We have the CPM stage thing, right? Which is kind of… the same. Um… So, the last tech design that they shared? Um, there is this, yeah."
},
{
"time": "16:02:48",
"speaker": "Michal Smaga",
"text": "Yeah, Max, just to clarify, I'm not up to date because, like, I'm still catching up after yesterday, so I haven't read the new tech design."
},
{
"time": "16:02:53",
"speaker": "Max Tsoy",
"text": "Yeah."
},
{
"time": "16:02:55",
"speaker": "Michal Smaga",
"text": "But, the goal is to, like, remove the old metrics like that, that were, like, very detailed and, like, very precise. Get rid of that just because of privacy reasons and, like, to implement the new ones."
},
{
"time": "16:03:01",
"speaker": "Max Tsoy",
"text": "Mm-hmm, mm-hmm."
},
{
"time": "16:03:04",
"speaker": "Alex Martemyanov",
"text": "Sweet. We may be not at the same page because I've just opened another tech design I wanted to discuss. And, uh, yes, so basically, uh, I suggest to remove, uh, all the old metrics and, uh, this recorder thing, and, uh, delegate, uh, proxy."
},
{
"time": "16:03:24",
"speaker": "Max Tsoy",
"text": "Okay."
},
{
"time": "16:03:28",
"speaker": "Alex Martemyanov",
"text": "And basically the only basic metric that would relate page committed state to the dashboard update."
},
{
"time": "16:03:37",
"speaker": "Max Tsoy",
"text": "Mm-hmm, mm-h."
},
{
"time": "16:03:38",
"speaker": "Alex Martemyanov",
"text": "Uh, message. So if we don't receive it within, uh, 30 seconds, uh, we…"
},
{
"time": "16:03:45",
"speaker": "Michal Smaga",
"text": "Record the stacks that you want."
},
{
"time": "16:03:45",
"speaker": "Alex Martemyanov",
"text": "So we expect to receive it. Yeah, but… before we navigate to another page, or before we close this tab. And if the page was alive for longer than 30 seconds, so basically it was measured, there were some cases when the dashboard update received after 30 seconds."
},
{
"time": "16:04:01",
"speaker": "Max Tsoy",
"text": "Mm-hmm."
},
{
"time": "16:04:06",
"speaker": "Alex Martemyanov",
"text": "It's for loading, because of, um… getting load, or whatever. And, uh, so if we navigate from the page, and, uh, it never received, uh, anything, and the page was active more than 30 seconds, we send a metric of the page of CPM hand. And if it was, uh, caught on another job."
},
{
"time": "16:04:22",
"speaker": "Max Tsoy",
"text": "Mm-hmm."
},
{
"time": "16:04:27",
"speaker": "Alex Martemyanov",
"text": "After it, we send the permanent breakage."
},
{
"time": "16:04:28",
"speaker": "Max Tsoy",
"text": "Right."
},
{
"time": "16:04:31",
"speaker": "Alex Martemyanov",
"text": "There was another issue of matching tab against actual… web extension tab against actual tab that we are measuring, because if we open the same page in two tabs, it basically can match both. Ahm. there is the tab index, and there is window ID that we cannot access from the app itself, but there is, like, a heuristics that allow us match it if we send all the window IDs. Um, so WebKit gives the window an ID in, uh… Consequently, it's not zero-based. It may be like 44, 72, and so on, but they are always ordered consequently as they are received from the app itself. And if we send these IDs from the script, we can match exact window and exact tab that the message is meant for."
},
{
"time": "16:05:32",
"speaker": "Max Tsoy",
"text": "Wait, wait, sorry, are you saying, uh… Are you saying we can match… we can map the extension tab ID to the native tab ID?"
},
{
"time": "16:05:39",
"speaker": "Alex Martemyanov",
"text": "Yes. Yes. Yes."
},
{
"time": "16:05:43",
"speaker": "Max Tsoy",
"text": "Oh, that's a, that's a great thing. I, uh, I."
},
{
"time": "16:05:45",
"speaker": "Alex Martemyanov",
"text": "So I was breaking my head the last three days on how could we actually match it to…"
},
{
"time": "16:05:45",
"speaker": "Michal Smaga",
"text": "Yeah, we…"
},
{
"time": "16:05:52",
"speaker": "Alex Martemyanov",
"text": "To avoid, uh, false positives."
},
{
"time": "16:05:52",
"speaker": "Max Tsoy",
"text": "We have a separate, like. Separate tests somewhere we have that, uh… But how would it work in a nutshell again?"
},
{
"time": "16:06:00",
"speaker": "Alex Martemyanov",
"text": "Yeah, basically we have a tab index that we set for the tab in the window."
},
{
"time": "16:06:07",
"speaker": "Max Tsoy",
"text": "Yes."
},
{
"time": "16:06:08",
"speaker": "Alex Martemyanov",
"text": "Oh, it's, uh…"
},
{
"time": "16:06:09",
"speaker": "Max Tsoy",
"text": "Huh?"
},
{
"time": "16:06:11",
"speaker": "Michal Smaga",
"text": "A native one."
},
{
"time": "16:06:13",
"speaker": "Alex Martemyanov",
"text": "the tab collection index. So it's not tab ID, but tab index. And we can send… we also have this tab index in the extension."
},
{
"time": "16:06:19",
"speaker": "Max Tsoy",
"text": "Okay."
},
{
"time": "16:06:25",
"speaker": "Alex Martemyanov",
"text": "And we don't need tab ID in that case. And it would just work on iOS, because we only have one window on iOS. But on macOS, there is another complication. If we open exact two windows with exact the same tab URLs in the same order."
},
{
"time": "16:06:33",
"speaker": "Max Tsoy",
"text": "Right, right."
},
{
"time": "16:06:41",
"speaker": "Alex Martemyanov",
"text": "There is a corner case that we cannot match it. But the window enumeration goes with the groan counter in the webkit. So it's not zero based, so it's not consequent. But it is growing."
},
{
"time": "16:07:01",
"speaker": "Max Tsoy",
"text": "Okay, uh, can you…"
},
{
"time": "16:07:02",
"speaker": "Alex Martemyanov",
"text": "So, a web extension window ID could be…"
},
{
"time": "16:07:04",
"speaker": "Max Tsoy",
"text": "Can you drop some message in this stack design, then? Because this is the…"
},
{
"time": "16:07:09",
"speaker": "Alex Martemyanov",
"text": "Both."
},
{
"time": "16:07:09",
"speaker": "Max Tsoy",
"text": "I think we considered this to… There, um…"
},
{
"time": "16:07:14",
"speaker": "Alex Martemyanov",
"text": "So basically the idea is to have another growing counter from the native side. And whenever we return a window, an extension window from the app, we grow the counter. So we will have window IDs starting like this. 1, 2, 3, and so on. And from WebKit, we would have IDs like 44, 72, 99, and so on. But there will be always two arrays of the matching size, and the sequence would match. So here we can match exactly window ID. from web extension and window ID that we gave it."
},
{
"time": "16:07:47",
"speaker": "Max Tsoy",
"text": "Okay. Um… Yes, I mean, I'm not, like, following 100%, but I think it… I mean, if there is a way, I would like, uh, if you can read that, uh, that tech design."
},
{
"time": "16:08:06",
"speaker": "Alex Martemyanov",
"text": "Okay, I will post an update there."
},
{
"time": "16:08:06",
"speaker": "Max Tsoy",
"text": "just see if… because I know we… I see we actually did consider, like, the indexes and stuff, but. Um, uh, yeah, just like consider if as long as all of these are also, um, considered, I think whatever. We can do to map it, it will really help, because it's also… it's a problem for other reasons as well, right? Because now we're just matched by this, like, domains, and it's just messy. Um…"
},
{
"time": "16:08:32",
"speaker": "Alex Martemyanov",
"text": "Yep."
},
{
"time": "16:08:38",
"speaker": "Max Tsoy",
"text": "Anyways, okay, so, but to the, um… Uh, do this, uh, metrics thing, right? So, I… I will look at it later. Right now. It sounds like the, uh, tech design from, uh, your tech design sounds, like, very similar to what we already have, the CPM stage parameter. Which is already there, and we didn't plan to remove it. Uh, so I will need to check that. Um, and the second… Uh, one is… so this other… tasks, the… remove temporary CPM breakage diagnostics. So it's assigned to me now. I don't know if it makes sense for you to cover that too. Or, uh, I should do, because some of it is an extension. Um… Because if your goal is to clean up the old metrics and make permanent ones. Maybe that's also… but it's okay, I mean, if… I can also do that. That's no problem."
},
{
"time": "16:09:49",
"speaker": "Alex Martemyanov",
"text": "I mean, I only wanted to validate the direction. There is one tech design for you for the need stage. I will check what you suggested. Another one is mostly for Michal, I guess."
},
{
"time": "16:09:55",
"speaker": "Max Tsoy",
"text": "Yeah."
},
{
"time": "16:10:05",
"speaker": "Alex Martemyanov",
"text": "But I just wanted to sync up with you both, so we have… Okay. the shared understanding of, uh, the remaining stages. So what I'll suggest is basically removing, uh, the fixes that didn't work out."
},
{
"time": "16:10:21",
"speaker": "Max Tsoy",
"text": "Mm-hmm."
},
{
"time": "16:10:21",
"speaker": "Alex Martemyanov",
"text": "We will keep updates on launch, base, stable base URLs restart after error six. But the ones that didn't appear in the metrics are just removed. I think it's kind of an obvious decision."
},
{
"time": "16:10:37",
"speaker": "Max Tsoy",
"text": "Mm-hmm. Mm-hmm."
},
{
"time": "16:10:40",
"speaker": "Alex Martemyanov",
"text": "And, uh, the next pull request, it just cleans up. all the metrics, so mostly on all the metrics that, uh, were added. And, uh, the next one would be re-adding based on some stabilized, uh, algorithms. So basically, we… rely on the sub or web page lifecycle, and if it didn't receive a CPM signal, we will report the breakage. So it should be that easy and nothing else gets connected."
},
{
"time": "16:11:10",
"speaker": "Max Tsoy",
"text": "Yes."
},
{
"time": "16:11:16",
"speaker": "Alex Martemyanov",
"text": "And it would be just a permanent metric that could give us understanding of where the trend going, but no reason."
},
{
"time": "16:11:16",
"speaker": "Max Tsoy",
"text": "Okay. Yep. Okay, so, so, so far, basically, what I'm hearing is that. Um… you want to implement the same thing as we have this CPM stage, but it will be a metric for all page loads, not just for, um, uh, breakage reports. Which is… good."
},
{
"time": "16:11:41",
"speaker": "Alex Martemyanov",
"text": "Uh, yes, uh, for all page loads, like a permanent metric, and uh…"
},
{
"time": "16:11:44",
"speaker": "Max Tsoy",
"text": "Yep."
},
{
"time": "16:11:46",
"speaker": "Alex Martemyanov",
"text": "So the init stage, it's kind of an extra mile for this effort just to. get an understanding of where it breaks. So it's not related, it's like another…"
},
{
"time": "16:11:56",
"speaker": "Max Tsoy",
"text": "Okay."
},
{
"time": "16:12:00",
"speaker": "Alex Martemyanov",
"text": "Extra magic."
},
{
"time": "16:12:01",
"speaker": "Max Tsoy",
"text": "Okay, so basically it's optional, this step design for the web extension lifecycle metrics."
},
{
"time": "16:12:02",
"speaker": "Alex Martemyanov",
"text": "Temporary. Yes."
},
{
"time": "16:12:08",
"speaker": "Max Tsoy",
"text": "Okay. Um… my first… my first reaction is that, like, why do we reinvent and, like, why don't we just reuse the same? thing, because the extension already sends the CPM stage, and we could just wire it up. So it doesn't just… so it's reported. also outside breakage reports. Um, but I haven't seen… haven't looked into the actual values, like… on the first glance, it looks the same. To me. Um, and then…"
},
{
"time": "16:12:42",
"speaker": "Alex Martemyanov",
"text": "Okay, I'll check and let you know."
},
{
"time": "16:12:44",
"speaker": "Max Tsoy",
"text": "Yeah, you can check that. Uh, and… Yeah, I will try to get back to you today. I'm a little tied up because we have the, uh, assessment today, and I need to… Sorry for that, but, um… I hope it's not blocking you too much."
},
{
"time": "16:13:04",
"speaker": "Alex Martemyanov",
"text": "Okay, no problem. I hope we can… get through it before Tuesday or Thursday."
},
{
"time": "16:13:14",
"speaker": "Max Tsoy",
"text": "Uh, no, but definitely before, before Tuesday, uh, I, uh, it's, like, today or tomorrow, we can resolve it, I'm sure."
},
{
"time": "16:13:22",
"speaker": "Alex Martemyanov",
"text": "Yep. That works."
},
{
"time": "16:13:25",
"speaker": "Max Tsoy",
"text": "Mm-hmm."
},
{
"time": "16:13:26",
"speaker": "Alex Martemyanov",
"text": "Okay, thanks. Then I'll give you an update async."
},
{
"time": "16:13:28",
"speaker": "Max Tsoy",
"text": "Right. Good route."
},
{
"time": "16:13:30",
"speaker": "Michal Smaga",
"text": "Alex, can you stay for a moment?"
},
{
"time": "16:13:32",
"speaker": "Alex Martemyanov",
"text": "Yep."
},
{
"time": "16:13:34",
"speaker": "Max Tsoy",
"text": "Uh, let's see if I can leave without…"
},
{
"time": "16:13:37",
"speaker": "Michal Smaga",
"text": "Okay, so we can…"
},
{
"time": "16:13:39",
"speaker": "Max Tsoy",
"text": "No, I can, I can, I can…"
}
]
}
@@ -14,3 +14,115 @@ source: "https://us01mynotes.zoom.us/DLwAbUQ_RZ6X_D63xiX_Ww"
- **15:53:25 · Max Tsoy:** Yeah, so I only have a few minutes, but, like, I have this question I asked about this dashboard yesterday. So, I don't understand… So, basically… We, uh, before this project, right, I made this, uh, this, uh, dashboard. With the… which was tracking this, uh, side effect. That kind of correlated with the error rate. And, uh, so we have this signal that shows. That at least one message was received from the extension, basically. Right. And it turned out that, like, I think it was, like, 10% of macOS breakage reports. Did not receive any message from the extension. So, which kind of meant that the… something was stuck. And I think that… that was the… Before this project, that was the best indicator of the problem. Right now, I'm looking at that dashboard, and I see that this… that signal only went up. on macOS. Uh, so… I don't understand, is it still relevant, or… do we, uh… I know we didn't look at it, but, like… Um… I'm not sure what it tells us. Does it actually mean that we didn't move anything, or…
- **15:54:54 · Michal Smaga:** Yeah. But the question is, like, the the signal is only for the braggage reports. Right?
- **15:55:00 · Max Tsoy:** Yes, but still, like, I would expect… it's… it's not… it's all breakage reports, right? So… Yes, it is filtered by people who report something, but it's still a signal and it's not related to what kind of breakage report it is.
- **15:55:21 · Michal Smaga:** And you filter by the macOS version as well, right?
- **15:55:24 · Max Tsoy:** Um… I think so. Yes, it's a stacked thing. It's in the same… in the same… Dashboards. Uh, this one. Right, so, uh, it's hidden under CPM diagnostics. Role.
- **15:55:51 · Alex Martemyanov:** Yeah, and just answered in your removed breakage diagnostic thread. And yeah, I basically asked Claude to check the breakages and related reports.
- **15:55:51 · Max Tsoy:** That's what I… Mm-hmm. Oh. Okay. Good.
- **15:56:06 · Alex Martemyanov:** So, it highlighted that, uh… The baseline is basically the same with the waiting signal, but there is a new report categories. Uh, that may be affecting, uh. that weighting metric and.
- **15:56:24 · Max Tsoy:** Okay, so…
- **15:56:27 · Alex Martemyanov:** So…
- **15:56:27 · Max Tsoy:** So we… we got more reports?
- **15:56:29 · Alex Martemyanov:** if there is a report flow on the error page, uh, which basically has no, uh, CPM running on it, and we've been affected, uh, with it with the, uh, CPM breakage pixels as well, uh, because, uh, they're not run on determination and there are pages on macOS.
- **15:56:37 · Max Tsoy:** Okay.
- **15:56:47 · Alex Martemyanov:** And if we exclude the report flow for the error page, it stays steady basically. And answering your treatment effect question, yeah, basically it was not…
- **15:56:53 · Max Tsoy:** Okay, okay.
- **15:57:02 · Alex Martemyanov:** lowering much, not affecting much the overall breakage signal magic.
- **15:57:08 · Max Tsoy:** Mm-hmm. Okay, so…
- **15:57:10 · Alex Martemyanov:** Because the main signal still stays in the seemingly normal flow as the metrics shown.
- **15:57:17 · Max Tsoy:** Right. Okay, so basically, the increase is probably driven by something else, but there was no decrease either. Um…
- **15:57:30 · Alex Martemyanov:** There was a… Probably less than a percent decrease and there's still fluctuation depending on a week measured, so we can tell for sure.
- **15:57:34 · Max Tsoy:** Yeah, yeah. Okay. Got it. Okay, so, uh, uh, alright, now, uh, I'm looking at your tech design, and my main question there is, do we… Do we not already have this? Because… so the tech design is basically proposing to send more, like, status reports from the extension.
- **15:58:04 · Alex Martemyanov:** Yep.
- **15:58:05 · Max Tsoy:** But we already have one signal, which is exactly the same thing, actually. So, if you look at this old… that design. that introduced that… other signal. So, the CPM dashboard state… Uh, specifically. Um… So basically, immediately after navigation, we set the CPM dashboard state to waiting in native code. And then… The first thing that… The extension does. After it gets… after it injects the content script. it's… initializes, and then… Almost immediately sends back the refresh CPM dashboard state. So… The problem is that. In the error state, this message is not received. So… Like, what you're proposing there… I think you will just not receive. Most of these messages. And that's the whole point of problem, right? So, like. Um… I don't think it's a problem of the granularity. It's basically that in this error state, we don't get. Any response?
- **15:59:37 · Alex Martemyanov:** Mmhm.
- **15:59:37 · Max Tsoy:** So, it kind of looks like the messaging is broken. And that's the whole reason, right? Like, we don't know what's happening. Is the extension stuck? Is it just the messaging that doesn't get through, or is it the… You know, the whole… Thing doesn't work.
- **15:59:55 · Alex Martemyanov:** I think it was mainly based on the, um… In the state that may be pending before we send the… The repair is unmarked state. Like, uh, for example, the config, uh, the config update state.
- **16:00:09 · Max Tsoy:** Right.
- **16:00:14 · Alex Martemyanov:** or. And. What else was there? Let me just open it.
- **16:00:21 · Max Tsoy:** Right, so basically you're suggesting that within that initialization, maybe it gets stuck. In between these first… so, what we could do…
- **16:00:28 · Alex Martemyanov:** Yes, yes.
- **16:00:32 · Max Tsoy:** We could just send one message. Immediately, like, on… On wake up. And I think we actually already do. Uh, don't we, like, there is some… I don't know, um, I don't remember exactly, but the… I think there was… so… What I'm saying is that I don't know if the, uh, all these other stages are… Um… useful, because I think it… the problem is that we… I suspect we won't get the first message, even.
- **16:01:07 · Alex Martemyanov:** Maybe, but… Probably we don't know.
- **16:01:15 · Max Tsoy:** So we have this… the first…
- **16:01:15 · Alex Martemyanov:** So here we kind of add no wait, so there is no await, it's kind of asynchronously as soon as we…
- **16:01:24 · Max Tsoy:** Mm-hmm.
- **16:01:26 · Alex Martemyanov:** Wake of the script wakes. Unless so first. Yeah, so there is the stage for, like, no, nothing was received, then, uh… H script in it. There may be possible hang on the config read, and if it hangs for whatever reason, it won't get through and won't get to the dashboard update state. Uh, so the suggestion is, like, sending, uh… And… log messages to… To set the list. Received state. that we could report in the main CPM breakage pixel.
- **16:02:15 · Max Tsoy:** Yeah, um…
- **16:02:16 · Alex Martemyanov:** I see.
- **16:02:18 · Max Tsoy:** Well, then I would… I would suggest just to trim it down to, like, the very first. signal, because… Um, I think for the rest, we actually… we have some… We have the CPM stage thing, right? Which is kind of… the same. Um… So, the last tech design that they shared? Um, there is this, yeah.
- **16:02:48 · Michal Smaga:** Yeah, Max, just to clarify, I'm not up to date because, like, I'm still catching up after yesterday, so I haven't read the new tech design.
- **16:02:53 · Max Tsoy:** Yeah.
- **16:02:55 · Michal Smaga:** But, the goal is to, like, remove the old metrics like that, that were, like, very detailed and, like, very precise. Get rid of that just because of privacy reasons and, like, to implement the new ones.
- **16:03:01 · Max Tsoy:** Mm-hmm, mm-hmm.
- **16:03:04 · Alex Martemyanov:** Sweet. We may be not at the same page because I've just opened another tech design I wanted to discuss. And, uh, yes, so basically, uh, I suggest to remove, uh, all the old metrics and, uh, this recorder thing, and, uh, delegate, uh, proxy.
- **16:03:24 · Max Tsoy:** Okay.
- **16:03:28 · Alex Martemyanov:** And basically the only basic metric that would relate page committed state to the dashboard update.
- **16:03:37 · Max Tsoy:** Mm-hmm, mm-h.
- **16:03:38 · Alex Martemyanov:** Uh, message. So if we don't receive it within, uh, 30 seconds, uh, we…
- **16:03:45 · Michal Smaga:** Record the stacks that you want.
- **16:03:45 · Alex Martemyanov:** So we expect to receive it. Yeah, but… before we navigate to another page, or before we close this tab. And if the page was alive for longer than 30 seconds, so basically it was measured, there were some cases when the dashboard update received after 30 seconds.
- **16:04:01 · Max Tsoy:** Mm-hmm.
- **16:04:06 · Alex Martemyanov:** It's for loading, because of, um… getting load, or whatever. And, uh, so if we navigate from the page, and, uh, it never received, uh, anything, and the page was active more than 30 seconds, we send a metric of the page of CPM hand. And if it was, uh, caught on another job.
- **16:04:22 · Max Tsoy:** Mm-hmm.
- **16:04:27 · Alex Martemyanov:** After it, we send the permanent breakage.
- **16:04:28 · Max Tsoy:** Right.
- **16:04:31 · Alex Martemyanov:** There was another issue of matching tab against actual… web extension tab against actual tab that we are measuring, because if we open the same page in two tabs, it basically can match both. Ahm. there is the tab index, and there is window ID that we cannot access from the app itself, but there is, like, a heuristics that allow us match it if we send all the window IDs. Um, so WebKit gives the window an ID in, uh… Consequently, it's not zero-based. It may be like 44, 72, and so on, but they are always ordered consequently as they are received from the app itself. And if we send these IDs from the script, we can match exact window and exact tab that the message is meant for.
- **16:05:32 · Max Tsoy:** Wait, wait, sorry, are you saying, uh… Are you saying we can match… we can map the extension tab ID to the native tab ID?
- **16:05:39 · Alex Martemyanov:** Yes. Yes. Yes.
- **16:05:43 · Max Tsoy:** Oh, that's a, that's a great thing. I, uh, I.
- **16:05:45 · Alex Martemyanov:** So I was breaking my head the last three days on how could we actually match it to…
- **16:05:45 · Michal Smaga:** Yeah, we…
- **16:05:52 · Alex Martemyanov:** To avoid, uh, false positives.
- **16:05:52 · Max Tsoy:** We have a separate, like. Separate tests somewhere we have that, uh… But how would it work in a nutshell again?
- **16:06:00 · Alex Martemyanov:** Yeah, basically we have a tab index that we set for the tab in the window.
- **16:06:07 · Max Tsoy:** Yes.
- **16:06:08 · Alex Martemyanov:** Oh, it's, uh…
- **16:06:09 · Max Tsoy:** Huh?
- **16:06:11 · Michal Smaga:** A native one.
- **16:06:13 · Alex Martemyanov:** the tab collection index. So it's not tab ID, but tab index. And we can send… we also have this tab index in the extension.
- **16:06:19 · Max Tsoy:** Okay.
- **16:06:25 · Alex Martemyanov:** And we don't need tab ID in that case. And it would just work on iOS, because we only have one window on iOS. But on macOS, there is another complication. If we open exact two windows with exact the same tab URLs in the same order.
- **16:06:33 · Max Tsoy:** Right, right.
- **16:06:41 · Alex Martemyanov:** There is a corner case that we cannot match it. But the window enumeration goes with the groan counter in the webkit. So it's not zero based, so it's not consequent. But it is growing.
- **16:07:01 · Max Tsoy:** Okay, uh, can you…
- **16:07:02 · Alex Martemyanov:** So, a web extension window ID could be…
- **16:07:04 · Max Tsoy:** Can you drop some message in this stack design, then? Because this is the…
- **16:07:09 · Alex Martemyanov:** Both.
- **16:07:09 · Max Tsoy:** I think we considered this to… There, um…
- **16:07:14 · Alex Martemyanov:** So basically the idea is to have another growing counter from the native side. And whenever we return a window, an extension window from the app, we grow the counter. So we will have window IDs starting like this. 1, 2, 3, and so on. And from WebKit, we would have IDs like 44, 72, 99, and so on. But there will be always two arrays of the matching size, and the sequence would match. So here we can match exactly window ID. from web extension and window ID that we gave it.
- **16:07:47 · Max Tsoy:** Okay. Um… Yes, I mean, I'm not, like, following 100%, but I think it… I mean, if there is a way, I would like, uh, if you can read that, uh, that tech design.
- **16:08:06 · Alex Martemyanov:** Okay, I will post an update there.
- **16:08:06 · Max Tsoy:** just see if… because I know we… I see we actually did consider, like, the indexes and stuff, but. Um, uh, yeah, just like consider if as long as all of these are also, um, considered, I think whatever. We can do to map it, it will really help, because it's also… it's a problem for other reasons as well, right? Because now we're just matched by this, like, domains, and it's just messy. Um…
- **16:08:32 · Alex Martemyanov:** Yep.
- **16:08:38 · Max Tsoy:** Anyways, okay, so, but to the, um… Uh, do this, uh, metrics thing, right? So, I… I will look at it later. Right now. It sounds like the, uh, tech design from, uh, your tech design sounds, like, very similar to what we already have, the CPM stage parameter. Which is already there, and we didn't plan to remove it. Uh, so I will need to check that. Um, and the second… Uh, one is… so this other… tasks, the… remove temporary CPM breakage diagnostics. So it's assigned to me now. I don't know if it makes sense for you to cover that too. Or, uh, I should do, because some of it is an extension. Um… Because if your goal is to clean up the old metrics and make permanent ones. Maybe that's also… but it's okay, I mean, if… I can also do that. That's no problem.
- **16:09:49 · Alex Martemyanov:** I mean, I only wanted to validate the direction. There is one tech design for you for the need stage. I will check what you suggested. Another one is mostly for Michal, I guess.
- **16:09:55 · Max Tsoy:** Yeah.
- **16:10:05 · Alex Martemyanov:** But I just wanted to sync up with you both, so we have… Okay. the shared understanding of, uh, the remaining stages. So what I'll suggest is basically removing, uh, the fixes that didn't work out.
- **16:10:21 · Max Tsoy:** Mm-hmm.
- **16:10:21 · Alex Martemyanov:** We will keep updates on launch, base, stable base URLs restart after error six. But the ones that didn't appear in the metrics are just removed. I think it's kind of an obvious decision.
- **16:10:37 · Max Tsoy:** Mm-hmm. Mm-hmm.
- **16:10:40 · Alex Martemyanov:** And, uh, the next pull request, it just cleans up. all the metrics, so mostly on all the metrics that, uh, were added. And, uh, the next one would be re-adding based on some stabilized, uh, algorithms. So basically, we… rely on the sub or web page lifecycle, and if it didn't receive a CPM signal, we will report the breakage. So it should be that easy and nothing else gets connected.
- **16:11:10 · Max Tsoy:** Yes.
- **16:11:16 · Alex Martemyanov:** And it would be just a permanent metric that could give us understanding of where the trend going, but no reason.
- **16:11:16 · Max Tsoy:** Okay. Yep. Okay, so, so, so far, basically, what I'm hearing is that. Um… you want to implement the same thing as we have this CPM stage, but it will be a metric for all page loads, not just for, um, uh, breakage reports. Which is… good.
- **16:11:41 · Alex Martemyanov:** Uh, yes, uh, for all page loads, like a permanent metric, and uh…
- **16:11:44 · Max Tsoy:** Yep.
- **16:11:46 · Alex Martemyanov:** So the init stage, it's kind of an extra mile for this effort just to. get an understanding of where it breaks. So it's not related, it's like another…
- **16:11:56 · Max Tsoy:** Okay.
- **16:12:00 · Alex Martemyanov:** Extra magic.
- **16:12:01 · Max Tsoy:** Okay, so basically it's optional, this step design for the web extension lifecycle metrics.
- **16:12:02 · Alex Martemyanov:** Temporary. Yes.
- **16:12:08 · Max Tsoy:** Okay. Um… my first… my first reaction is that, like, why do we reinvent and, like, why don't we just reuse the same? thing, because the extension already sends the CPM stage, and we could just wire it up. So it doesn't just… so it's reported. also outside breakage reports. Um, but I haven't seen… haven't looked into the actual values, like… on the first glance, it looks the same. To me. Um, and then…
- **16:12:42 · Alex Martemyanov:** Okay, I'll check and let you know.
- **16:12:44 · Max Tsoy:** Yeah, you can check that. Uh, and… Yeah, I will try to get back to you today. I'm a little tied up because we have the, uh, assessment today, and I need to… Sorry for that, but, um… I hope it's not blocking you too much.
- **16:13:04 · Alex Martemyanov:** Okay, no problem. I hope we can… get through it before Tuesday or Thursday.
- **16:13:14 · Max Tsoy:** Uh, no, but definitely before, before Tuesday, uh, I, uh, it's, like, today or tomorrow, we can resolve it, I'm sure.
- **16:13:22 · Alex Martemyanov:** Yep. That works.
- **16:13:25 · Max Tsoy:** Mm-hmm.
- **16:13:26 · Alex Martemyanov:** Okay, thanks. Then I'll give you an update async.
- **16:13:28 · Max Tsoy:** Right. Good route.
- **16:13:30 · Michal Smaga:** Alex, can you stay for a moment?
- **16:13:32 · Alex Martemyanov:** Yep.
- **16:13:34 · Max Tsoy:** Uh, let's see if I can leave without…
- **16:13:37 · Michal Smaga:** Okay, so we can…
- **16:13:39 · Max Tsoy:** No, I can, I can, I can…
@@ -19,6 +19,291 @@
"time": "16:15:28",
"speaker": "Michal Smaga",
"text": "Okay. So without matching, we may not see all the issues. Because it's, like…"
},
{
"time": "16:15:36",
"speaker": "Alex Martemyanov",
"text": "I don't… Yes, uh…"
},
{
"time": "16:15:39",
"speaker": "Michal Smaga",
"text": "Because if you have two tabs for the same domain, one may load, and if one fails, you will get, like, the the error won't be reported. Right?"
},
{
"time": "16:15:47",
"speaker": "Alex Martemyanov",
"text": "Yes, kind of it. So basically, we could keep the same matching algorithm if we receive it twice. for the same URL, just consider it healthy, and, uh, not report. It would… it would work, basically. So… But yeah, this is an extra mile."
},
{
"time": "16:16:07",
"speaker": "Michal Smaga",
"text": "And in the matching… And the matching would require, like, the web extension change as well, right?"
},
{
"time": "16:16:14",
"speaker": "Alex Martemyanov",
"text": "Yes. So, the changes to send the… to send the window ID and all the window IDs."
},
{
"time": "16:16:16",
"speaker": "Michal Smaga",
"text": "Just to report the…"
},
{
"time": "16:16:22",
"speaker": "Alex Martemyanov",
"text": "In the sequence that we've got them."
},
{
"time": "16:16:25",
"speaker": "Michal Smaga",
"text": "Yeah, yeah."
},
{
"time": "16:16:26",
"speaker": "Alex Martemyanov",
"text": "So we could match to arrays. in how the web extension IDs given in order, and how they were accessed from the upside in the same order, but different IDs."
},
{
"time": "16:16:35",
"speaker": "Michal Smaga",
"text": "Okay. Yeah, but this is kind of, like, an optional, or do you suggest doing that? Okay."
},
{
"time": "16:16:44",
"speaker": "Alex Martemyanov",
"text": "Yeah, yeah, this is an optional. I'm gonna comment in the design for matching, and it probably won't be included in the pull request I'm gonna open soon."
},
{
"time": "16:16:50",
"speaker": "Michal Smaga",
"text": "Okay."
},
{
"time": "16:16:56",
"speaker": "Alex Martemyanov",
"text": "So, currently, there is two… To ready ones."
},
{
"time": "16:17:00",
"speaker": "Michal Smaga",
"text": "Two PRs, yeah."
},
{
"time": "16:17:03",
"speaker": "Alex Martemyanov",
"text": "To be ours mostly or not to. I think it's only one, right? Second one is not there yet."
},
{
"time": "16:17:13",
"speaker": "Michal Smaga",
"text": "No, there are two."
},
{
"time": "16:17:15",
"speaker": "Alex Martemyanov",
"text": "There is two. Okay, probably the second one maybe not pushed. I'll just check a bit later."
},
{
"time": "16:17:22",
"speaker": "Michal Smaga",
"text": "There's remove diagnostics and remove ineffective fixes."
},
{
"time": "16:17:33",
"speaker": "Alex Martemyanov",
"text": "Yeah, let me just check it quickly. If it's up to date. Yeah, so remove, uh, remove fixes, the first one. It removes the feature flags and the fixes that weren't there, so it's mostly removal. It shouldn't be nothing complicated. Yeah, the second one is also actual. It removes the metrics. So, it's meant to be merged."
},
{
"time": "16:18:06",
"speaker": "Michal Smaga",
"text": "Okay, so… so there will be, like, a third… there will be a third one for the… for the permanent ones. Okay."
},
{
"time": "16:18:09",
"speaker": "Alex Martemyanov",
"text": "There will be third one re-adding. Yes, yes. So, and I'm…"
},
{
"time": "16:18:13",
"speaker": "Michal Smaga",
"text": "I've started setting up the testing for the PR, so I'll be reviewing it shortly, so we have it merged soon. Uh, yeah, other than that, I will catch up on the tech design and to see, like, uh, if I understood them correctly. And yeah."
},
{
"time": "16:18:31",
"speaker": "Alex Martemyanov",
"text": "Okay, okay, great. Thanks."
},
{
"time": "16:18:33",
"speaker": "Michal Smaga",
"text": "So do you have anything, like, blocking you for moving forward to the third one, or…"
},
{
"time": "16:18:40",
"speaker": "Alex Martemyanov",
"text": "Not really. Uh, so I was investigating, uh, how to… how to match it exactly, how far should we go. Probably we shouldn't, uh, go with it on the start PR."
},
{
"time": "16:18:40",
"speaker": "Michal Smaga",
"text": "We need to…"
},
{
"time": "16:18:52",
"speaker": "Alex Martemyanov",
"text": "And it could be totally optional."
},
{
"time": "16:18:56",
"speaker": "Michal Smaga",
"text": "Yeah. I I I would discover the the index thing for, like, it's good that you have, like, research. It's so, like, doing, like, some kind of, like, a brain dump of, like, the approach or, like, having, like, some spec for it. I think, maximum."
},
{
"time": "16:18:56",
"speaker": "Alex Martemyanov",
"text": "And, yeah."
},
{
"time": "16:19:13",
"speaker": "Michal Smaga",
"text": "Drop the link to, like, earlier discussion that we had, like, earlier this year, like, uh… We were figuring out that it wasn't, like, easily possible, so we we kind of, like, moved away from it. So maybe it would be worthy to to to, like, follow up it separately."
},
{
"time": "16:19:28",
"speaker": "Alex Martemyanov",
"text": "Mm-hmm."
},
{
"time": "16:19:29",
"speaker": "Michal Smaga",
"text": "Uh, on Tuesday, like, Russell was asking about the scope of the project and, like, when we finish up, so I would probably, like. Focus on like, like, like. Basically, focus on wrapping stuff up and, like, probably having a post-mortem, like, next week. So so I think this is, like, of a priority. And, yeah, and setting up the, like, the yeah. Just just wrap up the project, and probably we can do some additional work after that."
},
{
"time": "16:20:00",
"speaker": "Alex Martemyanov",
"text": "Yeah, okay. Okay, so I will just, my main question or my main point here was to think on the metric and how it works. So basically, we load a page, and uh… we report it after we close the tab, or if we navigate somewhere else. And if… The page was alive, was up for more than 30 seconds. So we had enough time for extension to load, and it didn't receive any CPM messages. It would be reported as broken. you… It should probably give us a general idea if how the trend is going."
},
{
"time": "16:20:43",
"speaker": "Michal Smaga",
"text": "And if you close the tab or navigate away within the 30 seconds, we don't report it, like, for nothing, right?"
},
{
"time": "16:20:52",
"speaker": "Alex Martemyanov",
"text": "Oh. I think so. Yeah."
},
{
"time": "16:20:59",
"speaker": "Michal Smaga",
"text": "I'll check… yeah, I'll try to give you the feedback on the tech design, so probably, like, this is something that we want to…"
},
{
"time": "16:20:59",
"speaker": "Alex Martemyanov",
"text": "So we can…"
},
{
"time": "16:21:07",
"speaker": "Michal Smaga",
"text": "like, clarify this week, so, like, probably Monday you can, like, implement the… Sorry. Eat precisely, and then we are good, I think."
},
{
"time": "16:21:18",
"speaker": "Alex Martemyanov",
"text": "Yeah, so it's most ready here locally, and I think today, later today, I'm going to open. Open it up and ping you and update the tech designs. So, yeah, there is some exception for the reporting, like a page or private pages."
},
{
"time": "16:21:30",
"speaker": "Michal Smaga",
"text": "Okay."
},
{
"time": "16:21:37",
"speaker": "Alex Martemyanov",
"text": "Alright. Or session restoration, or back navigation, or history navigation, so basically the ones that, uh, loaded from cache, and we had some… problems with them and race conditions."
},
{
"time": "16:21:50",
"speaker": "Michal Smaga",
"text": "Okay."
},
{
"time": "16:21:50",
"speaker": "Alex Martemyanov",
"text": "And then… But for general navigations, and yeah, so there's page lifetime, let's… It should be more than 30 seconds. If we had it two times in a row for different tabs, it means, like, permanent breakage of the extension."
},
{
"time": "16:22:07",
"speaker": "Michal Smaga",
"text": "And for the page restoration, do you detect it and act differently?"
},
{
"time": "16:22:07",
"speaker": "Alex Martemyanov",
"text": "Should give… Uh, it's me, uh… So we have this wait time before navigation, so we expect the extension to load. And it's limited for 5 seconds, but if it takes longer, it may still load after the page loads from cache, and it means it won't go through the page deload path and… Won't activate the extension. So, the… CPM won't be fired up and CUTIBRANCE won't be handled sometimes."
},
{
"time": "16:22:46",
"speaker": "Michal Smaga",
"text": "Mm-hmm. Yeah."
},
{
"time": "16:22:48",
"speaker": "Alex Martemyanov",
"text": "Thank you. It was, uh, like, our best effort, and it seems unreasonable to keep it… to keep it waiting longer than 5 seconds for users, uh, to extend this type of… this startup time. But there are still some reports. It seems like it went slightly down, but not really fixing all the cases. But those are not permanent, mostly."
},
{
"time": "16:23:18",
"speaker": "Michal Smaga",
"text": "Okay, yeah. Hmmm…"
},
{
"time": "16:23:25",
"speaker": "Alex Martemyanov",
"text": "Yeah, that should be it, I guess."
},
{
"time": "16:23:29",
"speaker": "Michal Smaga",
"text": "Okay. So, I'm off to to testing the PRs and and let you know."
},
{
"time": "16:23:33",
"speaker": "Alex Martemyanov",
"text": "Mm-hmm."
},
{
"time": "16:23:34",
"speaker": "Michal Smaga",
"text": "Cool. If you have anything I can help you with, uh, just feel free to ping me. And…"
},
{
"time": "16:23:35",
"speaker": "Alex Martemyanov",
"text": "Okay, thanks."
},
{
"time": "16:23:41",
"speaker": "Michal Smaga",
"text": "Maybe if there's, like, we need another discussion with Max or whatever, like, we can, like, chat later or tomorrow morning."
},
{
"time": "16:23:50",
"speaker": "Alex Martemyanov",
"text": "Sure, I'll ping him the task center. Thank you, Anthony."
},
{
"time": "16:23:53",
"speaker": "Michal Smaga",
"text": "Okay, cool. Thank you."
},
{
"time": "16:23:56",
"speaker": "Alex Martemyanov",
"text": "Okay. Yep. Thank you."
}
]
}
@@ -14,3 +14,60 @@ source: "https://us01mynotes.zoom.us/rJFn9D3lRJmnlCMSomhPsw"
- **16:14:12 · Michal Smaga:** read the text. designs yet. So I have, like, maybe a dumb question. Like, why is the the the, like, figuring out the specific top index and, like, matching the tops required for that?
- **16:14:23 · Alex Martemyanov:** Basically, I was trying to make it as stable as possible to… To make it, like, live permanently there. And, uh, as Max told, we have, uh, some problem on, or maybe false positives, uh, for, uh, unmatched tab, so the matching is done by URL, basically. And if you open the same tab twice, with the same URL, it would be ambiguous at that point. And uh… so in the previous metrics, it's used to… Um, reports, like, uh, healthy, if, uh, we have… two consequently awaiting tabs. So, one could be unhealthy, but we won't report it anyways if we receive the message twice. But to make it kind of 100% bulletproof, and as well in other places, if we match exactly the web extension tab against the actual tab, it would.
- **16:15:28 · Michal Smaga:** Okay. So without matching, we may not see all the issues. Because it's, like…
- **16:15:36 · Alex Martemyanov:** I don't… Yes, uh…
- **16:15:39 · Michal Smaga:** Because if you have two tabs for the same domain, one may load, and if one fails, you will get, like, the the error won't be reported. Right?
- **16:15:47 · Alex Martemyanov:** Yes, kind of it. So basically, we could keep the same matching algorithm if we receive it twice. for the same URL, just consider it healthy, and, uh, not report. It would… it would work, basically. So… But yeah, this is an extra mile.
- **16:16:07 · Michal Smaga:** And in the matching… And the matching would require, like, the web extension change as well, right?
- **16:16:14 · Alex Martemyanov:** Yes. So, the changes to send the… to send the window ID and all the window IDs.
- **16:16:16 · Michal Smaga:** Just to report the…
- **16:16:22 · Alex Martemyanov:** In the sequence that we've got them.
- **16:16:25 · Michal Smaga:** Yeah, yeah.
- **16:16:26 · Alex Martemyanov:** So we could match to arrays. in how the web extension IDs given in order, and how they were accessed from the upside in the same order, but different IDs.
- **16:16:35 · Michal Smaga:** Okay. Yeah, but this is kind of, like, an optional, or do you suggest doing that? Okay.
- **16:16:44 · Alex Martemyanov:** Yeah, yeah, this is an optional. I'm gonna comment in the design for matching, and it probably won't be included in the pull request I'm gonna open soon.
- **16:16:50 · Michal Smaga:** Okay.
- **16:16:56 · Alex Martemyanov:** So, currently, there is two… To ready ones.
- **16:17:00 · Michal Smaga:** Two PRs, yeah.
- **16:17:03 · Alex Martemyanov:** To be ours mostly or not to. I think it's only one, right? Second one is not there yet.
- **16:17:13 · Michal Smaga:** No, there are two.
- **16:17:15 · Alex Martemyanov:** There is two. Okay, probably the second one maybe not pushed. I'll just check a bit later.
- **16:17:22 · Michal Smaga:** There's remove diagnostics and remove ineffective fixes.
- **16:17:33 · Alex Martemyanov:** Yeah, let me just check it quickly. If it's up to date. Yeah, so remove, uh, remove fixes, the first one. It removes the feature flags and the fixes that weren't there, so it's mostly removal. It shouldn't be nothing complicated. Yeah, the second one is also actual. It removes the metrics. So, it's meant to be merged.
- **16:18:06 · Michal Smaga:** Okay, so… so there will be, like, a third… there will be a third one for the… for the permanent ones. Okay.
- **16:18:09 · Alex Martemyanov:** There will be third one re-adding. Yes, yes. So, and I'm…
- **16:18:13 · Michal Smaga:** I've started setting up the testing for the PR, so I'll be reviewing it shortly, so we have it merged soon. Uh, yeah, other than that, I will catch up on the tech design and to see, like, uh, if I understood them correctly. And yeah.
- **16:18:31 · Alex Martemyanov:** Okay, okay, great. Thanks.
- **16:18:33 · Michal Smaga:** So do you have anything, like, blocking you for moving forward to the third one, or…
- **16:18:40 · Alex Martemyanov:** Not really. Uh, so I was investigating, uh, how to… how to match it exactly, how far should we go. Probably we shouldn't, uh, go with it on the start PR.
- **16:18:40 · Michal Smaga:** We need to…
- **16:18:52 · Alex Martemyanov:** And it could be totally optional.
- **16:18:56 · Michal Smaga:** Yeah. I I I would discover the the index thing for, like, it's good that you have, like, research. It's so, like, doing, like, some kind of, like, a brain dump of, like, the approach or, like, having, like, some spec for it. I think, maximum.
- **16:18:56 · Alex Martemyanov:** And, yeah.
- **16:19:13 · Michal Smaga:** Drop the link to, like, earlier discussion that we had, like, earlier this year, like, uh… We were figuring out that it wasn't, like, easily possible, so we we kind of, like, moved away from it. So maybe it would be worthy to to to, like, follow up it separately.
- **16:19:28 · Alex Martemyanov:** Mm-hmm.
- **16:19:29 · Michal Smaga:** Uh, on Tuesday, like, Russell was asking about the scope of the project and, like, when we finish up, so I would probably, like. Focus on like, like, like. Basically, focus on wrapping stuff up and, like, probably having a post-mortem, like, next week. So so I think this is, like, of a priority. And, yeah, and setting up the, like, the yeah. Just just wrap up the project, and probably we can do some additional work after that.
- **16:20:00 · Alex Martemyanov:** Yeah, okay. Okay, so I will just, my main question or my main point here was to think on the metric and how it works. So basically, we load a page, and uh… we report it after we close the tab, or if we navigate somewhere else. And if… The page was alive, was up for more than 30 seconds. So we had enough time for extension to load, and it didn't receive any CPM messages. It would be reported as broken. you… It should probably give us a general idea if how the trend is going.
- **16:20:43 · Michal Smaga:** And if you close the tab or navigate away within the 30 seconds, we don't report it, like, for nothing, right?
- **16:20:52 · Alex Martemyanov:** Oh. I think so. Yeah.
- **16:20:59 · Michal Smaga:** I'll check… yeah, I'll try to give you the feedback on the tech design, so probably, like, this is something that we want to…
- **16:20:59 · Alex Martemyanov:** So we can…
- **16:21:07 · Michal Smaga:** like, clarify this week, so, like, probably Monday you can, like, implement the… Sorry. Eat precisely, and then we are good, I think.
- **16:21:18 · Alex Martemyanov:** Yeah, so it's most ready here locally, and I think today, later today, I'm going to open. Open it up and ping you and update the tech designs. So, yeah, there is some exception for the reporting, like a page or private pages.
- **16:21:30 · Michal Smaga:** Okay.
- **16:21:37 · Alex Martemyanov:** Alright. Or session restoration, or back navigation, or history navigation, so basically the ones that, uh, loaded from cache, and we had some… problems with them and race conditions.
- **16:21:50 · Michal Smaga:** Okay.
- **16:21:50 · Alex Martemyanov:** And then… But for general navigations, and yeah, so there's page lifetime, let's… It should be more than 30 seconds. If we had it two times in a row for different tabs, it means, like, permanent breakage of the extension.
- **16:22:07 · Michal Smaga:** And for the page restoration, do you detect it and act differently?
- **16:22:07 · Alex Martemyanov:** Should give… Uh, it's me, uh… So we have this wait time before navigation, so we expect the extension to load. And it's limited for 5 seconds, but if it takes longer, it may still load after the page loads from cache, and it means it won't go through the page deload path and… Won't activate the extension. So, the… CPM won't be fired up and CUTIBRANCE won't be handled sometimes.
- **16:22:46 · Michal Smaga:** Mm-hmm. Yeah.
- **16:22:48 · Alex Martemyanov:** Thank you. It was, uh, like, our best effort, and it seems unreasonable to keep it… to keep it waiting longer than 5 seconds for users, uh, to extend this type of… this startup time. But there are still some reports. It seems like it went slightly down, but not really fixing all the cases. But those are not permanent, mostly.
- **16:23:18 · Michal Smaga:** Okay, yeah. Hmmm…
- **16:23:25 · Alex Martemyanov:** Yeah, that should be it, I guess.
- **16:23:29 · Michal Smaga:** Okay. So, I'm off to to testing the PRs and and let you know.
- **16:23:33 · Alex Martemyanov:** Mm-hmm.
- **16:23:34 · Michal Smaga:** Cool. If you have anything I can help you with, uh, just feel free to ping me. And…
- **16:23:35 · Alex Martemyanov:** Okay, thanks.
- **16:23:41 · Michal Smaga:** Maybe if there's, like, we need another discussion with Max or whatever, like, we can, like, chat later or tomorrow morning.
- **16:23:50 · Alex Martemyanov:** Sure, I'll ping him the task center. Thank you, Anthony.
- **16:23:53 · Michal Smaga:** Okay, cool. Thank you.
- **16:23:56 · Alex Martemyanov:** Okay. Yep. Thank you.
@@ -19,6 +19,526 @@
"time": "22:05:00",
"speaker": "russell holt",
"text": "Can you explain NPS, Francis? It's Net Promoter something or other."
},
{
"time": "22:05:02",
"speaker": "FD",
"text": "And he had to. Yeah, net promoter score. So it's basically a a signal of loyalty. So it's basically saying how likely would you recommend this product to a friend? And… And so it's like a… It's like a substitute for retention, basically. Um, yeah, so… it's a proxy. So it's, um, since we don't actually measure retention, it's the, you know, it's like a measure that we can take. Which will proxy for retention, so… Yeah, given that… satisfaction with ad blocking and CPM and MPS all raised at the same time. I mean, we can't say for sure, but we can be fairly confident that those improvements helped improve overall satisfaction. And attention."
},
{
"time": "22:05:52",
"speaker": "russell holt",
"text": "Yeah, it's gone up 9 percentage points over the past year. I think that's pretty impressive. Yeah, and ad blocking in particular has gone up more than 10. Looks like we had just over 10 points since second quarter, so… That's, um… people… 63% of people in the survey rate. ad blocking on DuckDuckGo better than other browsers they use. That's up 10 points. That's pretty… You know, presumably the work we've done has actually moved that number. Who knows for sure, but… That's what we've been working on, so…"
},
{
"time": "22:06:44",
"speaker": "FD",
"text": "Yeah, great stuff."
},
{
"time": "22:06:44",
"speaker": "russell holt",
"text": "She feel good. She feel good about that. Um, okay, uh, Alex, why don't you… you put yourself on the agenda. Tell us what's happened… happening with the, um… with your project and help us decide what to do about it."
},
{
"time": "22:07:00",
"speaker": "Alex Martemyanov",
"text": "Uh, yep, so the, uh, CPM breakage, so I've, uh, summarized… You can find the link in the description. summarize the current project states and refresh the data as of today's state. It's the first day of the last public release measurements. Um, so, um… On the… on the last week progress, uh, I found out there was, uh, some, uh… Post positive sense for error pages for Mac OS and that. lowered our metrics, uh, the 30% on failures, and it is now comparable with iOS, so it's still, uh, higher than 1%, but it's not 33%, it's 1 point. 9 of permanent breakage to our daily."
},
{
"time": "22:07:58",
"speaker": "russell holt",
"text": "What, what's… What's an example of an error page here?"
},
{
"time": "22:08:04",
"speaker": "Alex Martemyanov",
"text": "Any any error page basically any navigation like Wi Fi is not available or. Or some other problems like domain does not exist. So on that, on that page is naturally."
},
{
"time": "22:08:16",
"speaker": "russell holt",
"text": "Really? Like, the new tab page?"
},
{
"time": "22:08:20",
"speaker": "Alex Martemyanov",
"text": "Yep."
},
{
"time": "22:08:21",
"speaker": "russell holt",
"text": "Like the new tab page?"
},
{
"time": "22:08:23",
"speaker": "Alex Martemyanov",
"text": "Yeah. No. Not not the new tab page. New tab page was not measured, but error pages, like, domain name is."
},
{
"time": "22:08:24",
"speaker": "russell holt",
"text": "Really?"
},
{
"time": "22:08:31",
"speaker": "Alex Martemyanov",
"text": "Incorrect."
},
{
"time": "22:08:32",
"speaker": "russell holt",
"text": "Oh, okay."
},
{
"time": "22:08:33",
"speaker": "Alex Martemyanov",
"text": "So on that pages, natural ACPM won't load. And those, I used to follow the metrics. And, yeah, the the effect was around thirty thirty percent, and, that dropped, around one percent of daily users."
},
{
"time": "22:08:42",
"speaker": "russell holt",
"text": "Okay."
},
{
"time": "22:08:52",
"speaker": "Alex Martemyanov",
"text": "as expected. Uh, another two fixes, uh, that were a work in progress were merged, so to speak, as well, but they go, uh, to public release only. The next Monday. So, those are… kind of reproduced fixes as well. Still not the main breakage point here. And I expect some observable… expect from those, but probably it won't cross the 1% line, uh, still. And, uh, so we got this, uh, largest group, Neat and Wake, as discussed. And, uh, so basically it shows, uh, normal regular flow, and, uh, it shares around 30 to 40% of, uh, all the failures. And yeah, around 30% of permanent hands. So we need to look into the extension life cycle itself, I guess. And that is the point of today's discussion. Still there is a cleanup planned for the project because there were a lot of fixes implemented and some of them didn't work out because they were kind of fixing things synthetically reproduced but not happening in production. Uh, so, uh, I'm planning to roll… roll back the fixes that are… ineffective. And, uh, we need to clean up the data metrics, because they are… they are sending large chunks of data that we probably won't need anymore. We already collected. Everything that we needed from the current state. But I think we still are going to need some permanent metrics for the feature breakage. The one that's going to be presented in the earlier. page, so we can… Observe the spikes or other stuff. And, uh, the suggestion is to modify. the extension itself to track each lifecycle step, basically. Where do we lose those communications? Whether it's the extension not loading, or it needs not. getting to the browser or other parts of the sequence? Because there are a couple of hypotheses, although they seem not… they should not cause permanent breakage, like, for example. If the config is the… config fetch from the web extension is hanging, it will cause hanging of the whole queue, but it should be reloaded by the worker after, like, half a minute. So it's not validated, and we have no idea where exactly it breaks because other other metrics are all good. And not representative here."
},
{
"time": "22:11:57",
"speaker": "russell holt",
"text": "Okay, so it sounds like…"
},
{
"time": "22:11:57",
"speaker": "Alex Martemyanov",
"text": "Uh, so my suggestion for…"
},
{
"time": "22:12:00",
"speaker": "russell holt",
"text": "Go ahead."
},
{
"time": "22:12:02",
"speaker": "Alex Martemyanov",
"text": "So my suggestion for this week is to clean up the fixes that didn't work and clean up the metrics and feature flags where needed or put some feature flags to permanent state and add some extra metrics. Then we put the project on hold because we have run out of all the hypothesis and current issues we know."
},
{
"time": "22:12:12",
"speaker": "russell holt",
"text": "Okay."
},
{
"time": "22:12:25",
"speaker": "Alex Martemyanov",
"text": "And after we start receiving the data."
},
{
"time": "22:12:26",
"speaker": "russell holt",
"text": "Well, I think you should postmortem. I think you should close it. Yeah, so this is the… um… This is a good point. Uh, to, uh…"
},
{
"time": "22:12:35",
"speaker": "Alex Martemyanov",
"text": "Yeah."
},
{
"time": "22:12:36",
"speaker": "russell holt",
"text": "I think post-mortem it. Rather than putting it on hold, because I don't know what the trigger would be for pulling it back off hold, other than, like. you know, maybe you look at the data. So, I agree with your plan here. I think it's… let me try to make sure I understand. So, there's a bunch of cleanup to do, which seems… You know, to kind of… Things that you have tried here. are still hanging out and need to be wrapped up or deleted or whatever."
},
{
"time": "22:13:05",
"speaker": "Alex Martemyanov",
"text": "Yep."
},
{
"time": "22:13:09",
"speaker": "russell holt",
"text": "Okay, so there's the cleanup. Then you want to make some of the temporary metrics permanent?"
},
{
"time": "22:13:17",
"speaker": "Alex Martemyanov",
"text": "I guess, so make… so we got… so we would have some metric that could indicate, like, in the… some future failures, if they happen. For example, something changes in macOS."
},
{
"time": "22:13:19",
"speaker": "russell holt",
"text": "Okay. Yeah, okay. Okay. And then there was something about the life cycle. I wasn't sure I totally followed that."
},
{
"time": "22:13:38",
"speaker": "Alex Martemyanov",
"text": "then there is two fixes that are merged, but getting released on Monday, and, uh, we'll need an extra day, uh, for data analysis before we can post-mortem, probably next Tuesday, I guess. And, uh, there is an extent."
},
{
"time": "22:13:47",
"speaker": "russell holt",
"text": "Okay. Next Tuesday, perfect."
},
{
"time": "22:13:52",
"speaker": "Alex Martemyanov",
"text": "Yep. And, uh, there is, uh, this extended lifecycle metrics, so basically wipe out the metrics that we were measuring, like the surroundings, uh, the events of the browser itself, uh, but dive into the extension itself and its, uh, events that it's, uh."
},
{
"time": "22:13:53",
"speaker": "russell holt",
"text": "Okay."
},
{
"time": "22:14:11",
"speaker": "Alex Martemyanov",
"text": "goes through just to to get an idea where it fails, at which exactly and at which step. Because otherwise, the big the largest group is, looks normal from from the data that we have."
},
{
"time": "22:14:20",
"speaker": "russell holt",
"text": "Okay. Okay, I think I follow that. Okay, that sounds good."
},
{
"time": "22:14:35",
"speaker": "Alex Martemyanov",
"text": "Okay. So I will communicate this, I'll communicate it, with Max and, probably have a tech design on how to."
},
{
"time": "22:14:36",
"speaker": "russell holt",
"text": "Next next Tuesday."
},
{
"time": "22:14:43",
"speaker": "Alex Martemyanov",
"text": "How to do it, and uh… Yeah, we'll try to complete it this week."
},
{
"time": "22:14:50",
"speaker": "russell holt",
"text": "Okay. Michal, you're still the advisor. You're on board."
},
{
"time": "22:14:56",
"speaker": "Michal Smaga",
"text": "Yeah, so, yeah, we synced earlier today, and this is, like, what we discussed, so I fully approve of the plan, and I think this is, like, the most reasonable."
},
{
"time": "22:15:00",
"speaker": "russell holt",
"text": "I'm good."
},
{
"time": "22:15:05",
"speaker": "Michal Smaga",
"text": "Way to go to use that we can drop up everything."
},
{
"time": "22:15:10",
"speaker": "russell holt",
"text": "Okay."
},
{
"time": "22:15:10",
"speaker": "Michal Smaga",
"text": "Yeah, great job, Alex."
},
{
"time": "22:15:14",
"speaker": "russell holt",
"text": "Awesome. Thank you, Alex."
},
{
"time": "22:15:14",
"speaker": "Alex Martemyanov",
"text": "Thanks."
},
{
"time": "22:15:17",
"speaker": "russell holt",
"text": "Okay. Hey, Marcus, what's happening with the top sides?"
},
{
"time": "22:15:21",
"speaker": "Marcos Gurgel",
"text": "Hey, yes. So let me post this link here. Uh, top sites went public to 5% starting yesterday, so… Let me share my screen. The numbers that I mean, bottom line is we need more time to make any definitive conclusions. But the numbers I want to focus for now, given that we don't even have 24 hours of credible data. are these here. Uh, so, unhandled rate between top 10K and other. So, how many top 10K were detected versus how many were unhandled? How many others were detected versus how many were unhandled? And… Right now, it differs wildly between US and EU. So, top 10K in US. for less than 24 hours gave us 16% unhandled. 40% unhandled in the EU, so I think these numbers are still gonna stabilize. I don't think we should take them as meaning anything yet. Just to give you another idea."
},
{
"time": "22:16:23",
"speaker": "russell holt",
"text": "What does it mean unhandled here?"
},
{
"time": "22:16:28",
"speaker": "Marcos Gurgel",
"text": "Unhandled means…"
},
{
"time": "22:16:28",
"speaker": "russell holt",
"text": "Individual pop-ups?"
},
{
"time": "22:16:31",
"speaker": "Marcos Gurgel",
"text": "Yes, it means, uh… It means that… We detected a pop up in using regex, but we didn't detect it with rules or heuristics."
},
{
"time": "22:16:43",
"speaker": "russell holt",
"text": "Okay."
},
{
"time": "22:16:45",
"speaker": "Marcos Gurgel",
"text": "So, yeah, fell through the cracks, basically. We see them in regex, but…"
},
{
"time": "22:16:48",
"speaker": "russell holt",
"text": "That's not 16."
},
{
"time": "22:16:51",
"speaker": "Marcos Gurgel",
"text": "Sorry, go ahead."
},
{
"time": "22:16:51",
"speaker": "russell holt",
"text": "That's not 16.5% of sites, that's 16.5% of…"
},
{
"time": "22:16:57",
"speaker": "Marcos Gurgel",
"text": "Detected. So any page that we think was there was a pop up. For 16% of them, we didn't find a pop up effectively and we didn't remove any pop ups. And… Same for other sites. So this would be the long tail. But I'm not… Very interested in this right now. I'm more interested in the top sites. Just to give you another idea of why we shouldn't… we should ignore all the numbers that came before, uh, because everything was in internal before. These were the numbers of pop ups being detected up until yesterday. So around. 20 to 30 per day. And now we're up to, you know, 30,000. So today was only the first meaningful day of data, and the releases happened. So macOS happened in the morning in the EU. But it happened in the evening in. Uh, in the EU for iOS. So, I would… I would guess that… if we look at the US numbers for yesterday, they're going to be closer to reality, because I think a lot of people were already asleep in the EU by the time the app came out. So, what else can I… can I infer from this? And by the way, these… these panels, I'm… I'm experimenting with them, I'm trying to figure out which one gives me the more… Useful number. So far, I'm not I don't think comparing them is is. is meaningful, so comparing the rate of top 10K versus the rate of others, I think it's more important to compare them with themselves, so this panel here shows me."
},
{
"time": "22:18:30",
"speaker": "russell holt",
"text": "Right."
},
{
"time": "22:18:39",
"speaker": "Marcos Gurgel",
"text": "how many… it essentially shows me the same as this table. So how many detected versus how many unhandled. for each bucket. So, and this is what… This is what I think would be the ideal outcome. that would result in the least amount of work in putting CPM in maintenance, which is, we see a very low number of unhandled top 10K."
},
{
"time": "22:19:07",
"speaker": "russell holt",
"text": "Yeah."
},
{
"time": "22:19:08",
"speaker": "Marcos Gurgel",
"text": "And we see a fairly higher number than others. I think that that's the ideal. Outcome, but I don't know if we're good."
},
{
"time": "22:19:13",
"speaker": "russell holt",
"text": "Prior than others. Other."
},
{
"time": "22:19:16",
"speaker": "Marcos Gurgel",
"text": "Yeah, so the long tail would have more unhandled pop ups than the top 10K."
},
{
"time": "22:19:24",
"speaker": "russell holt",
"text": "Mm-hmm. Yeah."
},
{
"time": "22:19:25",
"speaker": "Marcos Gurgel",
"text": "Uh, but, I mean, regardless of how they relate, I think having this number in under 20% is… would be a good result. I don't know if, I mean… It's a dice roll at this point. Yeah."
},
{
"time": "22:19:37",
"speaker": "russell holt",
"text": "Do we do we have it? Do we have the relation? Do we have it? That number? Do we understand its relationship to sites in the dataset?"
},
{
"time": "22:19:47",
"speaker": "Marcos Gurgel",
"text": "Uh… Let me see if I understand the question."
},
{
"time": "22:19:50",
"speaker": "russell holt",
"text": "Like, are sites that are, quote, covered… That we have rules for, are they still seeing unhandled pop-ups?"
},
{
"time": "22:19:58",
"speaker": "Marcos Gurgel",
"text": "Right. No, we don't have that because we don't know which sites exactly people are visiting. If I understand the question correctly. So we can't correlate… Yeah. We can't correlate a domain with…"
},
{
"time": "22:20:08",
"speaker": "russell holt",
"text": "Oh, yeah, and…"
},
{
"time": "22:20:12",
"speaker": "Marcos Gurgel",
"text": "That has a rule for it. Versus one that doesn't."
},
{
"time": "22:20:19",
"speaker": "russell holt",
"text": "Okay, so Conrad asks, why isn't it zero? Why shouldn't it be zero?"
},
{
"time": "22:20:26",
"speaker": "Marcos Gurgel",
"text": "Oh, yeah. It should. It should. It's just that I was expecting, I mean, honestly, because the total coverage is right now around 22 or 30%. I was expecting this number to be higher. So yes, it should be zero. But if it's less than the average coverage. Uh, I think we're good, because then that would mean that…"
},
{
"time": "22:20:52",
"speaker": "russell holt",
"text": "What percent? Do you know? Sorry. That would mean that what?"
},
{
"time": "22:20:56",
"speaker": "Marcos Gurgel",
"text": "Yeah, go ahead."
},
{
"time": "22:20:57",
"speaker": "russell holt",
"text": "No, I want to know what it means."
},
{
"time": "22:20:58",
"speaker": "Marcos Gurgel",
"text": "No, uh, because right now, uh… I think… Let me see what I think it means. It means that if this is below the general average for coverage. Then we need to focus more on the long tail. If this is hitting the average for coverage, then we need to focus more on top 10K, because."
},
{
"time": "22:21:17",
"speaker": "russell holt",
"text": "Okay. But we still…"
},
{
"time": "22:21:23",
"speaker": "Marcos Gurgel",
"text": "Because we can get it's."
},
{
"time": "22:21:24",
"speaker": "russell holt",
"text": "But we still want to get it to zero."
},
{
"time": "22:21:25",
"speaker": "Marcos Gurgel",
"text": "We can get more results. Yeah."
},
{
"time": "22:21:27",
"speaker": "russell holt",
"text": "Right, because every site that…"
},
{
"time": "22:21:27",
"speaker": "Marcos Gurgel",
"text": "Yes, absolutely."
},
{
"time": "22:21:30",
"speaker": "russell holt",
"text": "You know, ranked by popularity, presumably, every site that we then be sure that we. of properly completely covering has a bigger impact on that number then. Like, what's the approach for long tail? You pick."
},
{
"time": "22:21:42",
"speaker": "Marcos Gurgel",
"text": "Yeah."
},
{
"time": "22:21:47",
"speaker": "russell holt",
"text": "Just, you just go down the list or you pick them at random, you pick like. You have to do a lot more work in the long tail, right, than the top sites is what I'm kind of getting at."
},
{
"time": "22:21:59",
"speaker": "Marcos Gurgel",
"text": "Yeah, I agree. And to that end, even at the top sites, this question of, like, which… Which sites do we tackle first? we don't quite have the answer to that, right? But we can… we can make assumptions based on sites' popularity. But… With the current metrics, we don't have a way to see whether a specific site is causing… is skewing the numbers, right?"
},
{
"time": "22:22:23",
"speaker": "russell holt",
"text": "Sure, sure, but… Sure, but we have, you know, first of all, they are in the top 10K, so they're top 10K, based on our definition of that."
},
{
"time": "22:22:32",
"speaker": "Marcos Gurgel",
"text": "Okay."
},
{
"time": "22:22:34",
"speaker": "russell holt",
"text": "But then, you know, you can go through them in some order. Still, just doing them all is, to Conrad's point, kind of what we want to aspire to, I think."
},
{
"time": "22:22:39",
"speaker": "Marcos Gurgel",
"text": "Yes. Yeah, absolutely."
},
{
"time": "22:22:50",
"speaker": "russell holt",
"text": "Okay, dumb question. We send this for… Uh, do you know how many people have CPM… Disabled. And do we send the pixel for…"
},
{
"time": "22:23:00",
"speaker": "Marcos Gurgel",
"text": "No."
},
{
"time": "22:23:01",
"speaker": "russell holt",
"text": "for when. Do we only send it when a user has the feature enabled? Sorry, that's a dumb question, but we should know that."
},
{
"time": "22:23:09",
"speaker": "Marcos Gurgel",
"text": "I… I… I don't know the answer to that either, but I'm fairly certain that summary pixels only go out if the user has CPM enabled, but I'll double-check that after the meeting."
},
{
"time": "22:23:19",
"speaker": "russell holt",
"text": "Okay. Okay, all right."
},
{
"time": "22:23:22",
"speaker": "Marcos Gurgel",
"text": "Yeah, but I don't know the answer to that either."
},
{
"time": "22:23:27",
"speaker": "russell holt",
"text": "Okay, so what's your next step here?"
},
{
"time": "22:23:27",
"speaker": "Marcos Gurgel",
"text": "So… So, next step on… yeah, on this data… on this data is to sit and wait, but the other side of the project, which is adding the user-centric metric, I think is gonna give us…"
},
{
"time": "22:23:31",
"speaker": "russell holt",
"text": "I mean, we have to wait, get more data."
},
{
"time": "22:23:42",
"speaker": "Marcos Gurgel",
"text": "some more actionable numbers. Because we're gonna end up with a number of, uh… Number of pages. That a user visits per day. in the top 10K with unhandled pop-ups. So what's missing from this data right now is how often a user is seeing an unhandled pop up in the top 10K or another for that matter. So… So I'm hoping that once the rest of my project is done, we have a panel that says. Uh, number of unhandled pop-ups. per user per day. In buckets, so 0 to 5, or… 5 to 10. And then from that, we can… We can gauge… the annoyance that. these unhandled pop-ups are causing on each user, right? So if… and I think that would give us a more credible signal for attacking either top 10K or other, because if we see that top 10K is… is… Relatively low here. But it's high in number of pages, in number of unhandled per page visits. Then that contradicts what we're seeing here, right? So… Yeah, I think bottom line is, I don't know very much about this data yet. I want to sit on it, but I want to progress with the other side of the project, which is to add the per user per day metric on this."
},
{
"time": "22:25:29",
"speaker": "russell holt",
"text": "Yeah, uh… But that's still, I mean. You still want to get that 6… like, if that's 16… I mean, it's that…"
}
]
}
@@ -14,3 +14,107 @@ source: "https://us01mynotes.zoom.us/e5TYWRytRz2WwWSe68BSWQ"
- **22:03:13 · russell holt:** Having a good week? Alright, let me try to figure out where I am. Here's the agenda. Yeah, a few. Thank you for adding some items. Um… So I wanted to start by highlighting this. Quarterly survey. a quarterly browser survey if you haven't seen it. It has some interesting. results from a lot of things that we're working on. we've been working on. Versus the ratings versus other browsers. Often for Windows in particular. for ad blocking has gone way up. People seem to really like that. CPM has made some improvements there with cookie banners aren't blocked. Yeah. And too many cookies sent. So some stuff we've been working on and waiting for for a long time. Pretty exciting if you go check it out if you're interested. So, just wanted to mention that.
- **22:04:54 · FD:** Yeah, and just to add, I think… What's impressive to me is that NPS has also increased during the same time.
- **22:05:00 · russell holt:** Can you explain NPS, Francis? It's Net Promoter something or other.
- **22:05:02 · FD:** And he had to. Yeah, net promoter score. So it's basically a a signal of loyalty. So it's basically saying how likely would you recommend this product to a friend? And… And so it's like a… It's like a substitute for retention, basically. Um, yeah, so… it's a proxy. So it's, um, since we don't actually measure retention, it's the, you know, it's like a measure that we can take. Which will proxy for retention, so… Yeah, given that… satisfaction with ad blocking and CPM and MPS all raised at the same time. I mean, we can't say for sure, but we can be fairly confident that those improvements helped improve overall satisfaction. And attention.
- **22:05:52 · russell holt:** Yeah, it's gone up 9 percentage points over the past year. I think that's pretty impressive. Yeah, and ad blocking in particular has gone up more than 10. Looks like we had just over 10 points since second quarter, so… That's, um… people… 63% of people in the survey rate. ad blocking on DuckDuckGo better than other browsers they use. That's up 10 points. That's pretty… You know, presumably the work we've done has actually moved that number. Who knows for sure, but… That's what we've been working on, so…
- **22:06:44 · FD:** Yeah, great stuff.
- **22:06:44 · russell holt:** She feel good. She feel good about that. Um, okay, uh, Alex, why don't you… you put yourself on the agenda. Tell us what's happened… happening with the, um… with your project and help us decide what to do about it.
- **22:07:00 · Alex Martemyanov:** Uh, yep, so the, uh, CPM breakage, so I've, uh, summarized… You can find the link in the description. summarize the current project states and refresh the data as of today's state. It's the first day of the last public release measurements. Um, so, um… On the… on the last week progress, uh, I found out there was, uh, some, uh… Post positive sense for error pages for Mac OS and that. lowered our metrics, uh, the 30% on failures, and it is now comparable with iOS, so it's still, uh, higher than 1%, but it's not 33%, it's 1 point. 9 of permanent breakage to our daily.
- **22:07:58 · russell holt:** What, what's… What's an example of an error page here?
- **22:08:04 · Alex Martemyanov:** Any any error page basically any navigation like Wi Fi is not available or. Or some other problems like domain does not exist. So on that, on that page is naturally.
- **22:08:16 · russell holt:** Really? Like, the new tab page?
- **22:08:20 · Alex Martemyanov:** Yep.
- **22:08:21 · russell holt:** Like the new tab page?
- **22:08:23 · Alex Martemyanov:** Yeah. No. Not not the new tab page. New tab page was not measured, but error pages, like, domain name is.
- **22:08:24 · russell holt:** Really?
- **22:08:31 · Alex Martemyanov:** Incorrect.
- **22:08:32 · russell holt:** Oh, okay.
- **22:08:33 · Alex Martemyanov:** So on that pages, natural ACPM won't load. And those, I used to follow the metrics. And, yeah, the the effect was around thirty thirty percent, and, that dropped, around one percent of daily users.
- **22:08:42 · russell holt:** Okay.
- **22:08:52 · Alex Martemyanov:** as expected. Uh, another two fixes, uh, that were a work in progress were merged, so to speak, as well, but they go, uh, to public release only. The next Monday. So, those are… kind of reproduced fixes as well. Still not the main breakage point here. And I expect some observable… expect from those, but probably it won't cross the 1% line, uh, still. And, uh, so we got this, uh, largest group, Neat and Wake, as discussed. And, uh, so basically it shows, uh, normal regular flow, and, uh, it shares around 30 to 40% of, uh, all the failures. And yeah, around 30% of permanent hands. So we need to look into the extension life cycle itself, I guess. And that is the point of today's discussion. Still there is a cleanup planned for the project because there were a lot of fixes implemented and some of them didn't work out because they were kind of fixing things synthetically reproduced but not happening in production. Uh, so, uh, I'm planning to roll… roll back the fixes that are… ineffective. And, uh, we need to clean up the data metrics, because they are… they are sending large chunks of data that we probably won't need anymore. We already collected. Everything that we needed from the current state. But I think we still are going to need some permanent metrics for the feature breakage. The one that's going to be presented in the earlier. page, so we can… Observe the spikes or other stuff. And, uh, the suggestion is to modify. the extension itself to track each lifecycle step, basically. Where do we lose those communications? Whether it's the extension not loading, or it needs not. getting to the browser or other parts of the sequence? Because there are a couple of hypotheses, although they seem not… they should not cause permanent breakage, like, for example. If the config is the… config fetch from the web extension is hanging, it will cause hanging of the whole queue, but it should be reloaded by the worker after, like, half a minute. So it's not validated, and we have no idea where exactly it breaks because other other metrics are all good. And not representative here.
- **22:11:57 · russell holt:** Okay, so it sounds like…
- **22:11:57 · Alex Martemyanov:** Uh, so my suggestion for…
- **22:12:00 · russell holt:** Go ahead.
- **22:12:02 · Alex Martemyanov:** So my suggestion for this week is to clean up the fixes that didn't work and clean up the metrics and feature flags where needed or put some feature flags to permanent state and add some extra metrics. Then we put the project on hold because we have run out of all the hypothesis and current issues we know.
- **22:12:12 · russell holt:** Okay.
- **22:12:25 · Alex Martemyanov:** And after we start receiving the data.
- **22:12:26 · russell holt:** Well, I think you should postmortem. I think you should close it. Yeah, so this is the… um… This is a good point. Uh, to, uh…
- **22:12:35 · Alex Martemyanov:** Yeah.
- **22:12:36 · russell holt:** I think post-mortem it. Rather than putting it on hold, because I don't know what the trigger would be for pulling it back off hold, other than, like. you know, maybe you look at the data. So, I agree with your plan here. I think it's… let me try to make sure I understand. So, there's a bunch of cleanup to do, which seems… You know, to kind of… Things that you have tried here. are still hanging out and need to be wrapped up or deleted or whatever.
- **22:13:05 · Alex Martemyanov:** Yep.
- **22:13:09 · russell holt:** Okay, so there's the cleanup. Then you want to make some of the temporary metrics permanent?
- **22:13:17 · Alex Martemyanov:** I guess, so make… so we got… so we would have some metric that could indicate, like, in the… some future failures, if they happen. For example, something changes in macOS.
- **22:13:19 · russell holt:** Okay. Yeah, okay. Okay. And then there was something about the life cycle. I wasn't sure I totally followed that.
- **22:13:38 · Alex Martemyanov:** then there is two fixes that are merged, but getting released on Monday, and, uh, we'll need an extra day, uh, for data analysis before we can post-mortem, probably next Tuesday, I guess. And, uh, there is an extent.
- **22:13:47 · russell holt:** Okay. Next Tuesday, perfect.
- **22:13:52 · Alex Martemyanov:** Yep. And, uh, there is, uh, this extended lifecycle metrics, so basically wipe out the metrics that we were measuring, like the surroundings, uh, the events of the browser itself, uh, but dive into the extension itself and its, uh, events that it's, uh.
- **22:13:53 · russell holt:** Okay.
- **22:14:11 · Alex Martemyanov:** goes through just to to get an idea where it fails, at which exactly and at which step. Because otherwise, the big the largest group is, looks normal from from the data that we have.
- **22:14:20 · russell holt:** Okay. Okay, I think I follow that. Okay, that sounds good.
- **22:14:35 · Alex Martemyanov:** Okay. So I will communicate this, I'll communicate it, with Max and, probably have a tech design on how to.
- **22:14:36 · russell holt:** Next next Tuesday.
- **22:14:43 · Alex Martemyanov:** How to do it, and uh… Yeah, we'll try to complete it this week.
- **22:14:50 · russell holt:** Okay. Michal, you're still the advisor. You're on board.
- **22:14:56 · Michal Smaga:** Yeah, so, yeah, we synced earlier today, and this is, like, what we discussed, so I fully approve of the plan, and I think this is, like, the most reasonable.
- **22:15:00 · russell holt:** I'm good.
- **22:15:05 · Michal Smaga:** Way to go to use that we can drop up everything.
- **22:15:10 · russell holt:** Okay.
- **22:15:10 · Michal Smaga:** Yeah, great job, Alex.
- **22:15:14 · russell holt:** Awesome. Thank you, Alex.
- **22:15:14 · Alex Martemyanov:** Thanks.
- **22:15:17 · russell holt:** Okay. Hey, Marcus, what's happening with the top sides?
- **22:15:21 · Marcos Gurgel:** Hey, yes. So let me post this link here. Uh, top sites went public to 5% starting yesterday, so… Let me share my screen. The numbers that I mean, bottom line is we need more time to make any definitive conclusions. But the numbers I want to focus for now, given that we don't even have 24 hours of credible data. are these here. Uh, so, unhandled rate between top 10K and other. So, how many top 10K were detected versus how many were unhandled? How many others were detected versus how many were unhandled? And… Right now, it differs wildly between US and EU. So, top 10K in US. for less than 24 hours gave us 16% unhandled. 40% unhandled in the EU, so I think these numbers are still gonna stabilize. I don't think we should take them as meaning anything yet. Just to give you another idea.
- **22:16:23 · russell holt:** What does it mean unhandled here?
- **22:16:28 · Marcos Gurgel:** Unhandled means…
- **22:16:28 · russell holt:** Individual pop-ups?
- **22:16:31 · Marcos Gurgel:** Yes, it means, uh… It means that… We detected a pop up in using regex, but we didn't detect it with rules or heuristics.
- **22:16:43 · russell holt:** Okay.
- **22:16:45 · Marcos Gurgel:** So, yeah, fell through the cracks, basically. We see them in regex, but…
- **22:16:48 · russell holt:** That's not 16.
- **22:16:51 · Marcos Gurgel:** Sorry, go ahead.
- **22:16:51 · russell holt:** That's not 16.5% of sites, that's 16.5% of…
- **22:16:57 · Marcos Gurgel:** Detected. So any page that we think was there was a pop up. For 16% of them, we didn't find a pop up effectively and we didn't remove any pop ups. And… Same for other sites. So this would be the long tail. But I'm not… Very interested in this right now. I'm more interested in the top sites. Just to give you another idea of why we shouldn't… we should ignore all the numbers that came before, uh, because everything was in internal before. These were the numbers of pop ups being detected up until yesterday. So around. 20 to 30 per day. And now we're up to, you know, 30,000. So today was only the first meaningful day of data, and the releases happened. So macOS happened in the morning in the EU. But it happened in the evening in. Uh, in the EU for iOS. So, I would… I would guess that… if we look at the US numbers for yesterday, they're going to be closer to reality, because I think a lot of people were already asleep in the EU by the time the app came out. So, what else can I… can I infer from this? And by the way, these… these panels, I'm… I'm experimenting with them, I'm trying to figure out which one gives me the more… Useful number. So far, I'm not I don't think comparing them is is. is meaningful, so comparing the rate of top 10K versus the rate of others, I think it's more important to compare them with themselves, so this panel here shows me.
- **22:18:30 · russell holt:** Right.
- **22:18:39 · Marcos Gurgel:** how many… it essentially shows me the same as this table. So how many detected versus how many unhandled. for each bucket. So, and this is what… This is what I think would be the ideal outcome. that would result in the least amount of work in putting CPM in maintenance, which is, we see a very low number of unhandled top 10K.
- **22:19:07 · russell holt:** Yeah.
- **22:19:08 · Marcos Gurgel:** And we see a fairly higher number than others. I think that that's the ideal. Outcome, but I don't know if we're good.
- **22:19:13 · russell holt:** Prior than others. Other.
- **22:19:16 · Marcos Gurgel:** Yeah, so the long tail would have more unhandled pop ups than the top 10K.
- **22:19:24 · russell holt:** Mm-hmm. Yeah.
- **22:19:25 · Marcos Gurgel:** Uh, but, I mean, regardless of how they relate, I think having this number in under 20% is… would be a good result. I don't know if, I mean… It's a dice roll at this point. Yeah.
- **22:19:37 · russell holt:** Do we do we have it? Do we have the relation? Do we have it? That number? Do we understand its relationship to sites in the dataset?
- **22:19:47 · Marcos Gurgel:** Uh… Let me see if I understand the question.
- **22:19:50 · russell holt:** Like, are sites that are, quote, covered… That we have rules for, are they still seeing unhandled pop-ups?
- **22:19:58 · Marcos Gurgel:** Right. No, we don't have that because we don't know which sites exactly people are visiting. If I understand the question correctly. So we can't correlate… Yeah. We can't correlate a domain with…
- **22:20:08 · russell holt:** Oh, yeah, and…
- **22:20:12 · Marcos Gurgel:** That has a rule for it. Versus one that doesn't.
- **22:20:19 · russell holt:** Okay, so Conrad asks, why isn't it zero? Why shouldn't it be zero?
- **22:20:26 · Marcos Gurgel:** Oh, yeah. It should. It should. It's just that I was expecting, I mean, honestly, because the total coverage is right now around 22 or 30%. I was expecting this number to be higher. So yes, it should be zero. But if it's less than the average coverage. Uh, I think we're good, because then that would mean that…
- **22:20:52 · russell holt:** What percent? Do you know? Sorry. That would mean that what?
- **22:20:56 · Marcos Gurgel:** Yeah, go ahead.
- **22:20:57 · russell holt:** No, I want to know what it means.
- **22:20:58 · Marcos Gurgel:** No, uh, because right now, uh… I think… Let me see what I think it means. It means that if this is below the general average for coverage. Then we need to focus more on the long tail. If this is hitting the average for coverage, then we need to focus more on top 10K, because.
- **22:21:17 · russell holt:** Okay. But we still…
- **22:21:23 · Marcos Gurgel:** Because we can get it's.
- **22:21:24 · russell holt:** But we still want to get it to zero.
- **22:21:25 · Marcos Gurgel:** We can get more results. Yeah.
- **22:21:27 · russell holt:** Right, because every site that…
- **22:21:27 · Marcos Gurgel:** Yes, absolutely.
- **22:21:30 · russell holt:** You know, ranked by popularity, presumably, every site that we then be sure that we. of properly completely covering has a bigger impact on that number then. Like, what's the approach for long tail? You pick.
- **22:21:42 · Marcos Gurgel:** Yeah.
- **22:21:47 · russell holt:** Just, you just go down the list or you pick them at random, you pick like. You have to do a lot more work in the long tail, right, than the top sites is what I'm kind of getting at.
- **22:21:59 · Marcos Gurgel:** Yeah, I agree. And to that end, even at the top sites, this question of, like, which… Which sites do we tackle first? we don't quite have the answer to that, right? But we can… we can make assumptions based on sites' popularity. But… With the current metrics, we don't have a way to see whether a specific site is causing… is skewing the numbers, right?
- **22:22:23 · russell holt:** Sure, sure, but… Sure, but we have, you know, first of all, they are in the top 10K, so they're top 10K, based on our definition of that.
- **22:22:32 · Marcos Gurgel:** Okay.
- **22:22:34 · russell holt:** But then, you know, you can go through them in some order. Still, just doing them all is, to Conrad's point, kind of what we want to aspire to, I think.
- **22:22:39 · Marcos Gurgel:** Yes. Yeah, absolutely.
- **22:22:50 · russell holt:** Okay, dumb question. We send this for… Uh, do you know how many people have CPM… Disabled. And do we send the pixel for…
- **22:23:00 · Marcos Gurgel:** No.
- **22:23:01 · russell holt:** for when. Do we only send it when a user has the feature enabled? Sorry, that's a dumb question, but we should know that.
- **22:23:09 · Marcos Gurgel:** I… I… I don't know the answer to that either, but I'm fairly certain that summary pixels only go out if the user has CPM enabled, but I'll double-check that after the meeting.
- **22:23:19 · russell holt:** Okay. Okay, all right.
- **22:23:22 · Marcos Gurgel:** Yeah, but I don't know the answer to that either.
- **22:23:27 · russell holt:** Okay, so what's your next step here?
- **22:23:27 · Marcos Gurgel:** So… So, next step on… yeah, on this data… on this data is to sit and wait, but the other side of the project, which is adding the user-centric metric, I think is gonna give us…
- **22:23:31 · russell holt:** I mean, we have to wait, get more data.
- **22:23:42 · Marcos Gurgel:** some more actionable numbers. Because we're gonna end up with a number of, uh… Number of pages. That a user visits per day. in the top 10K with unhandled pop-ups. So what's missing from this data right now is how often a user is seeing an unhandled pop up in the top 10K or another for that matter. So… So I'm hoping that once the rest of my project is done, we have a panel that says. Uh, number of unhandled pop-ups. per user per day. In buckets, so 0 to 5, or… 5 to 10. And then from that, we can… We can gauge… the annoyance that. these unhandled pop-ups are causing on each user, right? So if… and I think that would give us a more credible signal for attacking either top 10K or other, because if we see that top 10K is… is… Relatively low here. But it's high in number of pages, in number of unhandled per page visits. Then that contradicts what we're seeing here, right? So… Yeah, I think bottom line is, I don't know very much about this data yet. I want to sit on it, but I want to progress with the other side of the project, which is to add the per user per day metric on this.
- **22:25:29 · russell holt:** Yeah, uh… But that's still, I mean. You still want to get that 6… like, if that's 16… I mean, it's that…