Embracing AI in Engineering: A Journey Towards AI-Native Practices with Yenny Cheung, VP at Bluefish

0:00
Hey Annie.
0:02
Hi. How’s it going?
0:04
Good. Welcome to the Swim podcasts. Uh
0:06
thank you for having me.
0:07
Good to see you again. Uh we’ve met I
0:10
don’t know if you remember we met at a
0:11
lead event in I think in Germany or in
0:14
London. I’m not sure.
0:15
Yeah, that’s right.
0:17
And since then I’ve been following your
0:19
career very interesting moves and
0:21
companies. Um and now you’re at Bluefish
0:25
AI.
0:26
That’s right.
0:27
Can you tell us a bit more about
0:28
yourself? like a few a few few seconds
0:31
of introduction and about Bluefish and
0:33
what you’re focused on at the moment.
0:36
Yeah, definitely. My name is Yiani
0:37
Chang. I’m currently VP of engineering
0:39
at Bluefish AI. What we do, we’re the
0:42
Fortune 500 marketing platform and we
0:45
are hinged upon the AI channels. So our
0:48
product provide visibility,
0:50
favorability, different set of metrics
0:52
to help our customers find out how
0:55
they’re fairing in the new GEO 8 world
0:58
when previously it’s just SEO. we can
1:00
see that search is really changing the
1:03
landscape dramatically um over the past
1:06
um two three years right so we’re well
1:08
positioned to capture that um
1:11
opportunity and for myself my career I
1:14
spent my time working for B2B category
1:18
creating companies scaling companies
1:21
from series A all the way to series D
1:24
plus and so I’ve seen a couple of times
1:27
firsthand what kind of engineering bets
1:29
is necess neessary for what dish and
1:32
bluefish is a very exciting opportunity.
1:34
So happy to tell you more about what
1:37
we’re trying out.
1:39
Very cool. Very interesting. So in in
1:41
the prep discussion we had, you told me
1:44
you’re already 40 engineers at Bluefish,
1:48
right? Growing fast. And that AI is
1:50
already your biggest code contributor.
1:52
Um
1:54
so I’m interested to know what does it
1:56
mean in terms of your day-to-day? How
1:58
does it actually look like for people
2:00
who are you know moving and evol
2:02
evolving towards AI nativeness? How does
2:05
it look how how does it look like?
2:08
Yeah, the team has grown really fast
2:11
from 12 engineers at that time. We have
2:13
one big team to 40 and we have several
2:16
teams now with their own uh workflow,
2:19
right? So a lot of the AI usage is
2:22
already baked into the SDLC in the
2:25
software development life cycle. So from
2:28
like all the way from product and design
2:31
taking up problem areas doing
2:33
prototyping getting early feedback to
2:36
the way we would do um uh specs right
2:41
together
2:42
both on the product side as well as
2:44
engineering side to use that as a
2:46
companion right to define how good would
2:49
look like for these documents all the
2:51
way to feeding that in you know
2:53
specdriven development into code
2:56
generation. ation into um development as
2:59
well as to code review and then also in
3:02
terms of deployment and we also adopt
3:04
some of the practices there. So I would
3:07
say day-to-day there are multiple touch
3:08
points for AI but it’s done in a way
3:11
where it helps us work right instead of
3:15
us working for the AI that’s how we
3:17
would want to view it. If you’re already
3:19
there, that’s already a great progress.
3:22
I think a lot of us are still working
3:24
for AI. So, uh, good to learn about
3:26
that. Uh, so when did you realize it
3:29
wasn’t just about, you know, adding new
3:31
tools in the SDLC, but
3:33
actually changing how the whole team,
3:36
right, thinks and works.
3:38
M so very early on in the AI hype uh
3:45
there’s talk especially from the
3:46
investment side right that AI is going
3:49
to just replace our jobs or you and I as
3:52
well right um like it’s really
3:55
exaggerated you know the whole company
3:57
now run by one person or you know half a
4:00
person and um so from that thinking
4:04
from an engineering mindset for me is
4:06
always thinking about that what what
4:09
problem set is this asantic coding
4:12
paradigm trying to solve and what are
4:16
the areas at which it can automate and
4:18
do a good job of and what not right
4:21
so quite soon enough in blue vision in
4:24
our organization along with technical
4:27
leadership we see that the opportunity
4:30
there is still the same as how we would
4:32
do engineering management before but
4:35
from a different perspective what I mean
4:37
is that the problems have always been
4:39
there. For example, code review, quality
4:42
control or how can we make sure that we
4:44
have better documentation? How can we
4:47
make sure we have um the the code uh the
4:50
code that we have written matches with
4:52
the spec uh spec that we have laid out
4:54
to solve, right? Or the business
4:56
requirements. These are problems that
4:58
exist in the past, but now there is just
5:00
an entire way to solving it. Now instead
5:03
of just a point solution that happens
5:06
when there are smaller pockets of um
5:09
innovation and disruption, this one
5:12
feels like it’s actually changing the
5:14
way work can be done at all of these
5:17
stages. So that’s why that’s how we
5:19
approach that thinking but it actually
5:21
doesn’t deviate too much from the way
5:24
traditional engineering management would
5:26
think about in solving problems and
5:28
doing it in such a way that we look at
5:30
bottlenecks. Right.
5:32
All right. Um, so let’s talk about and
5:35
we’ll go back into the actual process of
5:38
evolving towards these processes, but
5:41
I’m I’m curious about the mindset
5:44
and the cultural aspects of this shift
5:46
because you know you’re you’re quite a
5:49
new newer like a new company but uh
5:51
still this whole new SDLC is is is
5:55
rather new as well. Uh so were there any
5:57
folks uh at Bluefish on your team who
6:00
couldn’t make the shift or needed help
6:02
in that transition?
6:04
Yeah. Um
6:06
I would say um the DNA of Bluefish helps
6:11
with change fairly well and that’s also
6:14
one thing we have done well in how we
6:17
have created the culture and how we have
6:19
hired. So intrinsically folks are open
6:22
to that. That said, like we’re also very
6:25
conscious in creating a culture where we
6:28
use AI and not AI uses us. I’ll
6:30
elaborate that a little bit more. Um,
6:36
you know, the class of thinking of AI
6:38
just replaces our job, right? Right.
6:40
I think in a way that feels like we’re
6:42
just giving AI control and let’s see
6:43
what happens and let it find out, right?
6:46
But our way is actually more controlled
6:48
and the way we think about AI is how can
6:51
we actually hit the same set of KPI but
6:53
with better tooling.
6:55
Right? So in this uh frame of um in this
7:00
framework of thinking then I see less
7:04
that their folks might not be able to
7:06
catch up to the AI wave but more like do
7:08
people have the level of curiosity and
7:12
also the uh tolerance for change to
7:15
learn new things and that’s something
7:18
that I would have required of all
7:19
engineers and not specifically just for
7:21
AI.
7:22
Yeah for sure. uh do you have also like
7:24
existential discussions about the role
7:27
developers and you know this kind of
7:28
like stress of like FOMO of the new
7:31
technologies you know coming up every
7:33
other week.
7:36
Yeah. like whenever we have a team
7:37
meeting that comes up um you know is
7:40
there a world where we can really let AI
7:43
do end to-end development without as
7:46
much human in the loop and discussions
7:47
like that where we have landed on at the
7:50
moment is more um we for code that goes
7:55
into production we still need human in
7:58
the loop and we still need to have that
8:00
discipline to make sure quality is high
8:02
with human input but then we have also
8:04
created some pockets For example, we do
8:07
demos or we do prototypes and things
8:09
that are more separated out in the
8:11
platform or in the application that
8:13
we’re willing to okay then we put in
8:16
some vcoded material in there provided
8:19
that it doesn’t interact with like
8:21
compliance or security concern.
8:24
So that’s how we think about that.
8:26
Understood. So um when we talked
8:30
previously you told me about different
8:32
stages right of evolution of how do you
8:35
take an engineering organizations toward
8:37
this process of evolving towards AI
8:40
native nativeness. So if you were to
8:43
break it down into these stages what
8:45
would be the crawl walk run if you will
8:47
of AI modernizations for engineering
8:49
teams
8:50
and what does it look like?
8:53
Yeah. Um
8:55
for me the crawl phase would be earlier
9:00
usage of AI tooling and you have many
9:03
different versions happening locally
9:05
right oh people want to try a cursor
9:07
okay we use cursor someone use quad code
9:10
the other one uh uses co copilot for
9:13
example
9:14
and people are experimenting and trying
9:17
to find the norm of what works well and
9:20
what doesn’t um but there isn’t any
9:23
rules around that or a centralization
9:26
I think and then for the walk phase this
9:29
starts to come into being where we start
9:32
establishing new rules right okay we see
9:35
that a skill is useful for code review
9:38
let’s do one for TypeScript one for
9:40
Python we bake in all our uh thinking of
9:42
how good looks like into that so that we
9:44
have some standardization we agree to
9:46
use that as a team in part of the
9:48
process and we can agree with that in
9:51
within multiple teams not just within
9:53
one teams process and then for run I
9:56
would say this scope expands also to
9:58
other departments for example with um we
10:02
also work closely with our business
10:04
teams right they also advise our
10:06
customers they’re very close and they do
10:08
automation so how can we also establish
10:11
a framework of governance with other
10:13
departments now they also want to find
10:15
code some apps right what do we do and
10:17
what’s our role in engineering and to
10:19
figure out that step
10:21
how do how often do you break and rebild
10:23
built some of these processes that
10:25
you’re now establishing. Is it like a
10:27
very fast iterations or you have time to
10:30
see it coming and then
10:32
it’s very fast.
10:33
If you ask me like what everybody’s like
10:36
um process is like I don’t think I have
10:38
a good answer like we still have
10:40
different pockets like some in the phase
10:42
one the crawl phase some you know some
10:45
processes are more in the walk phase
10:48
already right we have some agreement and
10:50
then we’re still figuring out the run
10:52
step
10:53
yeah and that’s brings us to another
10:55
questions which is measurement in
10:57
general right we we want our
10:59
orgs to be measured and have like clear
11:02
KPIs And so how do you actually measure
11:05
that? How do you know it’s working?
11:07
Yeah, this is still a pretty early
11:09
discipline. One thing that I um one of
11:12
the frameworks that I saw uh being quite
11:15
helpful is one from DX. So they break
11:19
down the measurements into three bigger
11:22
buckets. That’s also how I would think
11:23
about measuring that within the
11:25
organization. One is on the utilization.
11:28
So how many people actually use the
11:30
tool? It’s pretty simple like we look at
11:32
the co-contribution as well. Um but that
11:35
is more a leading indicator to see okay
11:37
there’s some degree of adoption and
11:41
another one is impact that one actually
11:43
doesn’t deviate too much from Dora that
11:47
we historically have I still track the
11:49
same set of KPI um on um deployment
11:54
frequency on uh change failure rate um
11:57
on that set of metrics uh plus a few
12:00
additional ones for example we do have
12:03
uh monthly these surveys asking our
12:05
engineers how much time does AI uh
12:08
improve your productivity. So far we
12:10
have some findings only 25 I know only
12:14
uh 10% of our engineers mentioned the uh
12:18
productivity gain is under 25%. So I
12:21
think people view that like pretty
12:23
favorably and the last set of metrics
12:25
would be on cost. That’s also something
12:27
that we’re keeping uh more of an eye on.
12:30
there’s this token maxing phenomenon
12:33
also saw the situation at Meta and
12:35
[laughter]
12:36
maybe some other companies uh you know
12:39
struggling with um this this uh pillar
12:42
now that’s something that we’re also
12:44
monitoring we definitely see the cost
12:46
going up there um so we are not any
12:49
we’re not implementing much control yet
12:51
but that’s something to monitor
12:53
right um so I want to combine two
12:56
questions they had and one one of it you
12:59
already answered somehow is about the
13:02
the the human judgment in all that
13:04
process, right? So where where is it
13:06
still very important and pro and
13:08
prominent for you and also with the pace
13:11
of things, right? Code is going up very
13:14
fast. It’s being written very fast. How
13:16
do you keep things from going out of
13:18
control if there is a way to to call it
13:21
this way?
13:22
Yeah, that’s uh that’s also something
13:24
that is um pretty much on our minds. The
13:28
way I think about this um AI sometimes
13:32
lacks the taste like when you’re writing
13:35
right when we’re writing like for
13:36
example publishing to LinkedIn will we
13:39
do that first iteration and just put in
13:43
yeah right exactly and it’s the same for
13:46
code as well and so all of these
13:50
measures that we’re doing to give it an
13:52
MD or to you know have a code review
13:55
skill is to uh tune or help the AI
14:00
acquire that bit of the taste. And you
14:03
can do a SharePoint presentation in 15
14:06
seconds, right, with with Claude, but
14:08
what comes out of it is often times a
14:10
bit tasteless in my opinion, right? It’s
14:13
like 80% correct, but like that 20% is
14:16
just not there. So that’s that’s what
14:19
we’re um focusing on. And I still think
14:21
that that adjustment of taste and that
14:25
deviation of 20% is what we try to
14:28
minimize, right, when we use AI tooling.
14:30
I I recently saw a post on social media
14:32
saying we’re in the worst era of design
14:35
because everybody’s using AI generated
14:37
designs and all the like the flyers look
14:39
like uh [laughter]
14:42
a bit right like imagine that applied to
14:44
code as well. But we did do some
14:46
adjustment in the process. So previously
14:50
we have a more free flowing RFC process
14:53
where okay I trust the engineer to pick
14:57
the right people review and so on but
15:00
gradually I also noticed that every day
15:03
we started having two three RFCs
15:05
actually [laughter]
15:05
like the speed is quite fast and like
15:09
you say sometimes it also starts
15:11
spinning out of control. So now we we
15:14
are creating a bit better governance to
15:16
make sure we have a review process a bit
15:19
more in place to make sure that we
15:22
actually spend a bit more time on that
15:24
step and we can save more time actually
15:27
in the coding part because that’s not
15:29
really the bottleneck is solving the
15:30
right problem in the right way and that
15:32
is something I mean AI can help you
15:34
flesh out the RFC but you need to have a
15:37
clear thinking to give to the AI to help
15:39
you do that. Right.
15:41
All right. So part of the podcast idea
15:44
is also for having you know and as
15:47
you’re talking I’m thinking about other
15:49
engineering managers who are you know
15:51
who will be watching and learning from
15:52
that. Um so we’re we’re all trying to
15:55
like learn one from the other. Um if
15:58
there is something that you would say in
16:00
terms of uh guaranteed way of an
16:03
engineering team would fail right in
16:06
their quest for AI modernization.
16:09
uh what not to do, right? That can be a
16:12
managerial aspect of how to deal with
16:15
people or a process that you’ve tried
16:18
and it’s like a an assured failure. Do
16:21
you have something in mind for that?
16:24
I think
16:27
um I wouldn’t let the face of just let
16:32
everybody do what they want go for a
16:35
very long time. M
16:36
I do think that engineering management’s
16:38
role is to facilitate the storming phase
16:41
of teams. Right now I see that as a
16:43
storming phase, right? People figuring
16:45
out how to best use the tool in the most
16:47
productive way. So our role is to
16:50
provide that guidance. We should be
16:52
setting up this this governance
16:54
framework and set out uh guidance on
16:57
what is a good thing to do and what is a
16:58
not so good thing to do quite fast and
17:01
react to it very fast.
17:02
Right? you know heck
17:04
it’s finding the balance between you
17:06
know giving freedom to your people and
17:08
being innovative and so on and having
17:11
the frameworks that keeps things in
17:13
order in a way am I wrong
17:15
right like if we have a two three people
17:17
team then yeah maybe every day can be
17:19
vibe coding and you can iterate very
17:21
fast everybody can just put things into
17:23
the product we’re people and we serve
17:26
fortune 500 they cannot just see a
17:28
vipcoded feature out of nowhere with no
17:31
communication with other teams. So we
17:34
still have we have to kind of recreate
17:36
the process as we go because the speed
17:39
of things happening is faster. Then how
17:41
can we make sure we still communicate to
17:44
the customer what value they’re getting
17:46
from our features faster in a more
17:47
automated way or we have seen some
17:50
business teams now also doing their apps
17:52
to help with their productivity. How
17:55
much governance like what how do we
17:56
differentiate the ownership pattern? How
17:59
much engineering should be responsible
18:01
for? how much they should be responsible
18:03
for maintaining also the applications
18:05
that they build. These are things that
18:07
we need to set governance framework on
18:10
day after day then right and not just
18:12
let it for okay just let it run loose
18:15
right and see see what happens right I
18:16
think that is not a not the the greatest
18:18
idea to let that out for too long
18:21
and you told me you actually hired
18:23
someone full-time to help drive this
18:25
process and so can you let tell us what
18:28
you know what made you realize you
18:30
needed that person
18:32
um what what’s their role how’s their
18:34
impact is being measured
18:37
Yeah. Um so I have seen a couple of
18:40
cases of that in some companies for
18:43
example as for example in cloud now and
18:46
also other teams I I know they started
18:49
having an AI platform or AI productivity
18:53
team to more full-time dedicated to
18:57
automation and developer productivity.
19:01
And that’s an idea that um I’m
19:05
I’m I feel very positive about because
19:08
that allows the person to have the time
19:11
and space to think deeply about you know
19:13
how do we set up the right incentive
19:15
framework that we discussed right uh
19:18
often times if it’s afterthought it
19:20
doesn’t quite work and hence we open up
19:22
this role um and I’ve seen that trend
19:26
actually picking up as well now what are
19:29
they supposed to do is to I I can even
19:33
share right in our interview process we
19:35
did a path to production exercise
19:37
where they talk about in their past
19:39
experience
19:40
with AI or without how they have
19:42
optimized the flow for engineers.
19:45
So it’s really a developer productivity
19:47
DX type of role um that we’re looking
19:50
for. Now, last time I also alluded to
19:53
that hiring this person has been
19:56
challenging and I was very particular
19:59
about like the kind of profile I’m
20:00
looking for because on one hand this
20:03
person needs to have change management
20:06
experience.
20:08
You’re we’re setting rules as we go and
20:10
we oftenimes that means we need to say
20:12
no or that means we need to come up with
20:14
new strategies on how to navigate
20:15
situations and create possibilities.
20:19
It’s you know there’s a lot of um nuance
20:22
into handling people collaboration and
20:25
how to convince other people. So this
20:28
person ideally should have management
20:30
experience or at least strong leadership
20:32
sense but at the same time they also
20:34
have to be really hands-on [laughter]
20:37
because then you got to you know
20:39
implement solutions as you go and you
20:42
cannot just rely on other people to do
20:44
that for you. So that’s what we
20:46
explicitly looked for and yeah we we
20:49
found someone there
20:50
but we didn’t [laughter]
20:52
I think when I talked to the recruiter
20:54
she’s like you
20:56
do you really expect we find [laughter]
20:58
someone like that?
20:59
Okay I guess you knew what you wanted.
21:02
Um so what are the thing and I am
21:05
assuming there are many of those but
21:06
what are the things that you haven’t the
21:08
thing you haven’t figured out yet in
21:10
this change is there something on your
21:12
mind specifically that you are
21:14
struggling with
21:16
yeah I think um now we’re trying to
21:21
tackle the open point on how our
21:24
positioning should be with regards to
21:27
other departments innovation and the
21:30
applications and automations they built.
21:33
So internally I’m confident like over
21:36
time we keep on doing this bottleneck
21:39
analysis we keep cracking automation
21:41
that’s something engineers are
21:42
historically good at doing and we have
21:45
hired some strong foundation you know uh
21:48
engineers with strong foundation so I’m
21:49
not too worried but the one that we
21:51
really need to create maybe new new
21:54
innovation on how we would govern is
21:57
with other business teams
21:59
in no other world possible before that
22:03
they could have built apps themselves
22:06
without maybe sometimes talking to
22:08
product engineering. They just have
22:09
solutions. [laughter]
22:10
So how are we you know how how can we
22:14
one best support them but at the same
22:16
time also
22:19
um help us to make sure that we reduce
22:22
risk in terms of security in terms of
22:24
compliance right so we we need to have a
22:28
degree of um uh some some uh some
22:35
like we we need to be able to know
22:37
what’s going on right and be part of
22:39
that process but at the same time in a
22:40
non-blocking way that I don’t think that
22:43
we have a very clear answer across the
22:46
industries on how to do that at scale
22:48
yet and that’s something we’re actively
22:50
trying to figure out right
22:51
that’s super interesting what you’re
22:52
saying basically you have much more than
22:55
40 engineers in the whole organization
22:58
[laughter]
22:59
all right interesting all right to like
23:01
to to end this discussion you started by
23:05
telling us about your uh you know your
23:07
career path so [clears throat] now It’s
23:09
interesting for me to like have your two
23:11
cents on how your your job your
23:15
engineering manager you know is evolving
23:18
and how do you personally
23:20
take this change and you know now you
23:22
have to manage people and AI and
23:25
processes to rethink everything you’ve
23:27
learned right the same way engineers
23:30
need to like rebuild themselves around
23:32
these new technologies there is a
23:34
rebuild of the engineering manager in
23:36
terms of what do they need to manage
23:39
and what to care about. So can you like
23:41
tell us a bit more about that in a
23:43
philosophical way? [laughter]
23:47
I I can always talk philosophical.
23:50
So like there maybe there are two two
23:52
perspective right one is in terms of the
23:54
engineering culture we want to create.
23:56
Second is how we use AI in our personal
23:58
workflow and also helping us us become
24:00
more productive. So on the first um area
24:06
I think more of our attention actually
24:10
get put into
24:12
how can we incentivize the right things
24:14
to happen in the organization
24:17
because we see a lot in the news um
24:22
like one organization goes completely AI
24:24
pill and then uh you know almost like uh
24:29
every everyone is like forced to do AI
24:31
in a certain way and or maybe even doing
24:34
a lot of labeling to to get this to
24:36
work, right? So there are like some
24:38
extremes as well and having AI taking
24:42
over. I think our approach as of the
24:45
current moment right is still how can we
24:48
encourage engineers to do good work and
24:52
we are deliberately not prescriptive as
24:55
to what tooling they should be using but
24:57
we provide everything like we have
24:58
learning sessions we have guides on how
25:01
to set up all the MCPS everything is out
25:03
there for people to adopt but in the end
25:06
we still want to emphasize good work
25:08
business outcome outcome is the most
25:11
important and that’s something we really
25:12
want emphasize because of this taste
25:16
topic we we discussed
25:18
right like we we I don’t care about like
25:19
whether how much token well unless you
25:22
really blow it up right how much token
25:24
you’ve used but in the end is it good
25:26
work right that’s what we care about and
25:28
we spend a lot of time thinking about
25:29
like how do we best convey that right
25:33
um so that’s one thing
25:35
what is good what is taste that’s
25:37
interesting questions that you have on
25:39
your mind all the time
25:41
right because it can also exacerbate the
25:44
bad. I think that’s what AI does, right?
25:47
So, how can we emphasize the good and
25:49
paint a picture of how good looks like
25:51
and that’s almost like too agnostic. But
25:55
of course, I’m very bullish about the
25:56
good use of AI can help people become
25:59
good, right? But setting the right bar
26:02
and communicating that bar remains our
26:04
focus. And the second uh area is how can
26:07
we become more productive then? Yeah, we
26:09
also use a lot of MCP. I I also have I
26:13
have a daily digest now um uh inspired
26:16
by uh James Dier’s uh personal
26:19
newsletter. I think he mentioned some
26:21
advice advisory council or some
26:24
automation. I also have that where it
26:26
gives me the top five things I should
26:28
pay attention to in email in Renola in
26:31
GitHub in Jira. Like I don’t have enough
26:34
hours in a day to look at the work of 40
26:37
engineers anymore. like 40 engineers
26:40
five years ago is very different from
26:42
today.
26:43
So that has helped me uh have a better
26:46
grasp of the organization and that has
26:48
also helped our uh ems in you know
26:52
creating processes and communicating.
26:56
All right, Yenni, thank you so much for
26:58
joining the podcast. Uh I have
27:00
personally learned a lot through your
27:02
journey. Um, and I think it’s going to
27:05
be we could do the the same podcast 3
27:07
months from now, probably different
27:08
answers and [laughter]
27:11
they probably will be like, “No, this is
27:13
all nonsense now. This new era
27:16
now everything is recorded so we can
27:17
check.” Uh,
27:18
all right. Sounds good.
27:20
Thank you so much and see you soon.
27:23
Thank you as well. Thanks for having me.