Guys, welcome back to my channel. In this video, let's talk about one of the most challenging product questions in data science interviews, and that is how to improve a product. You may have better ways to answer this kind of question, and i'd love to hear them. There are three reasons that I consider this kind of question, most challenging. The first reason is that there are many different ways to ask questions. This kind of question can be very open ended such as how to input twitter. It can also be very specific such as how to increase what's on your mind posting on facebook. All this kind of questions can be categorized as how to improve product.
各位，欢迎回到我的频道。在本视频中，我们将讨论数据科学面试中最具挑战性的产品问题之一，即如何改进产品。你可能有更好的方法来回答这类问题，我很乐意听听。我认为这类问题最具挑战性的原因有三个。第一个原因是有许多不同的提问方式。这类问题可以是非常开放的，比如如何输入Twitter。它也可以是非常具体的，比如如何增加你在Facebook上发布的想法。所有这类问题都可以归类为如何改进产品。
Many reasons that I consider this kind of question challenging is that the answer should be different for different interviewers. What I by that is if you interview with the product manager, the answer should be different when you interview with a data scientist. Also, the answer will be different based on the type of the interview. If you ask this kind of question during a, technical phone screen, your answer should be short and to the .. But if you ask this kind question during onside interview, you want to spend more time. Can I find the question interact with the interweaver and provide more in depth answer?
我认为这类问题具有挑战性的许多原因是，不同的面试官应该有不同的答案。我想说的是，如果你面试的是产品经理，那么当你面试的是数据科学家时，答案应该是不同的。此外，根据面试的类型，答案也会有所不同。如果你在一个技术电话屏幕上问这类问题，你的回答应该简短。但如果你在现场采访中问这种问题，你会想花更多的时间。我可以找到问题，与交织器互动，并提供更深入的答案吗？
The third reason that I consider this kind of question challenging is that even though there are many ways to answer this kind of question, there are many ways to improve products. The interviewers are still looking for structured answers, basically, even if you have some golden ideas to improve product. But if your answer is unstructured or random, it's very hard for you to get some stuff from the interviewer. Without the deal. Let's see some simple questions. How to improve you the retention on twitter, how to improve user engagement on facebook, how to improve, what's on your mind posting on facebook, how to improve what's up? What feature would you add to it? So for the rest of the video, i'm gonna provide you with my friend works or a few steps. And I typically follow to answer this kind of question. I also use one interview question from facebook as an example to provide some simple answers to give me a sense of what the answer would look like.
我认为这类问题具有挑战性的第三个原因是，尽管有很多方法可以回答这类问题，但也有很多方法可以改进产品。面试官仍然在寻找结构化的答案，基本上，即使你有一些金点子来改进产品。但如果你的回答是无组织的或随机的，你就很难从面试官那里得到一些东西。没有交易。让我们看看一些简单的问题。如何提高你在Twitter上的留存率，如何提高Facebook上的用户参与度，如何改进，你在Facebook上发布的想法，如何改进？你会给它添加什么功能？在接下来的视频中，我将为您提供我朋友的作品或几个步骤。我通常会跟着回答这类问题。我还以Facebook上的一个面试问题为例，提供了一些简单的答案，让我对答案有一个大致的了解。
That's I the first step is to clarify the question. This is very similar to other kind of product questions we talk about before. It's important to narrow down the scope at the minimum. We should be clear about the high level goal of the improvement, meaning is it to improve engagement, retention, or revenue? Also, if the company has multiple products, we want to narrow down the scope to a specific product or feature we want to focus on. For example, if the question is to ask you to improve twitter, you can say something like twitter has a diverse set of product features such as fees, trees and others, which one shall we focus on?
第一步是澄清问题。这与我们之前讨论的其他类型的产品问题非常相似。最小限度地缩小范围很重要。我们应该明确改进的高层次目标，即是提高参与度、留存率还是收入？此外，如果公司有多个产品，我们希望将范围缩小到我们想要关注的特定产品或功能。例如，如果问题是要求你改进Twitter，你可以这样说：Twitter有一套多样化的产品功能，如费用、树和其他，我们应该关注哪一个？
Lastly, if the question is on a specific feature, for instance, how to improve the words online posting on facebook, we need to clarify how the future actually works, is the future only enabled for certain users or for all users. Do users have to love me in order to use that feature? One thing was knowing is that your interviews you may not fully come on the interviewer to give you a direction. Some interviewers may not give you any suggestion or feedback. So be prepared that the inter river refuses to be helpful. We as candidates need to have a clear idea on what is the goal of the improvement and which product to focus on. As that means that I still encourage you to clarify the question. One reason is that sometimes the interviewers to help you, they may provide you with more clarity for the question. Another reason is that it's critical to avoid miscommunication during interviews. What interviewers mean to say is not always what they actually said. If you assume you understand the question and you start answering in the wrong direction is a huge red flag.
最后，如果问题是关于一个特定的功能，例如，如何改进Facebook上的在线帖子，我们需要澄清未来实际上是如何运作的，未来是只为某些用户启用还是为所有用户启用。用户必须爱我才能使用该功能吗？有一件事是知道的，那就是在你的面试中，你可能不会完全让面试官给你一个方向。有些面试官可能不会给你任何建议或反馈。所以要做好准备，因特河拒绝提供帮助。作为候选人，我们需要清楚地知道改进的目标是什么，以及重点关注哪种产品。因为这意味着我仍然鼓励你澄清这个问题。其中一个原因是，有时面试官会帮助你，他们可能会为你提供更清晰的问题。另一个原因是，在面试过程中避免沟通不畅是至关重要的。面试官想说的并不总是他们实际所说的。如果你认为你理解了这个问题，而你开始从错误的方向回答，这是一个巨大的危险信号。
The next step after try find the question is to explain the approach, you will take to identify product improvement opportunities. There are many ways that you could use to come up with ideas to improve product. If you are a frequent user of a product, you may already have some ideas to improve it. Here, I just want to share with you two ways that I typically used to bring some ideas. Remember that we're in the brainstorm stage, you can be as outrageous as you want.
尝试发现问题后的下一步是解释方法，您将采取该方法来确定产品改进机会。有很多方法可以用来提出改进产品的想法。如果你经常使用某个产品，你可能已经有了一些改进它的想法。在这里，我只想和大家分享两种我通常用来带来一些想法的方式。记住，我们正处于头脑风暴阶段，你可以像你想的那样出格。
One way to come up with ideas to improve product is to analyze current user journeyman. Basically, when you learn on website, what would you do as a user? And then think about how we can reduce or remove friction in current user experience. Let's use increasing the number of what's on your mind posting on facebook, as an example. After long into facebook, in order for us to use the feature, we need to be aware of it. We could think about how often do people say it is the surface area large enough? Do people hover over the component? Some ideas we may use to increase people's awareness? One is to just increase the size of the component and the button, the larger the component, the higher chances people will see it. Also, after users logging, we could use a pop window to show this feature. This could also increase awareness. Another idea is to send emails or push notifications to people, to remind them the future exists.
提出改进产品的想法的一种方法是分析当前用户熟练工。基本上，当你在网站上学习时，作为一个用户，你会怎么做？然后思考我们如何减少或消除当前用户体验中的摩擦。让我们以增加你在Facebook上发布的想法的数量为例。在进入Facebook很长一段时间后，为了让我们使用这个功能，我们需要了解它。我们可以想一想，人们有多少次说它的表面积足够大？人们是否将鼠标悬停在组件上？我们可以用一些想法来提高人们的意识。一种是只增加组件和按钮的大小，组件越大，人们看到它的机会就越高。此外，在用户登录后，我们可以使用弹出窗口来显示此功能。这也可以提高认识。另一个想法是向人们发送电子邮件或推送通知，提醒他们未来的存在。
Now we're aware of the future. What do we do? Next we click on it and a window pops up. Then there are different scenarios. It's possible that we open the window, but we don't know what to say. So we end up not posting anything. It can reflect in the data that some users spend some time on the page and did not enter any text, then the abandoned for this kind of users. Think about how we can reduce friction to incentify them to post. You may have bad ideas, but here I want to share with you some of mine. We could probably provide a few tablets that can be used easily by users. For example, we can have a template that ask the user to fill in their feelings like I feel something today because of something else. Or if it's a special day, such as an anniversary or birthday of the user, or for someone they are closely connected to, we could pregenerate the content to simplify the process of posting.
现在我们意识到了未来。我们该怎么做？接下来我们点击它，弹出一个窗口。然后有不同的场景。有可能我们打开了窗户，却不知道该说什么。所以我们最终没有发布任何东西。它可以在数据中反映出一些用户在页面上花费了一些时间，并且没有输入任何文本，那么这类用户就被抛弃了。想想我们如何减少摩擦来激励他们发帖。你可能有不好的想法，但在这里我想和你分享我的一些想法。我们可能会提供一些用户可以轻松使用的平板电脑。例如，我们可以有一个模板，要求用户填写他们的感觉，就像我今天因为其他事情而感觉到一些事情。或者，如果这是一个特殊的日子，比如用户的周年纪念日或生日，或者是与他们关系密切的人，我们可以预先生成内容，以简化发布过程。
Now we have some ideas to reduce friction for users who have the intention to start, but don't know what to say. There are also users who know exactly what to say and then type in some words, but they end up banning the post. Those users are just one step further to posting. Let's see how we can reduce fiction here. To encourage them to post. We could remind them in a few hours or in a few days by sending email or notification, saying that, hey, you have a draft, would you like to post it directly then? They can just click button to post it. We can also provide option for more alice, say something like, would you like to continue editing and then provide the link back to the draft to continue the analysis of the user journey?
现在我们有一些想法，可以为那些有意向开始，但不知道该说什么的用户减少摩擦。还有一些用户知道该说什么，然后输入一些单词，但他们最终禁止了帖子。这些用户只是在发帖方面又前进了一步。让我们看看如何在这里减少虚构。鼓励他们发帖。我们可以在几个小时或几天内通过发送电子邮件或通知来提醒他们，说，嘿，你有一个草稿，你愿意直接发布它吗？他们只需点击按钮即可发布。我们还可以为Alice提供更多选项，比如，您是否愿意继续编辑，然后提供返回到草稿的链接，以继续分析用户旅程？
Another scenario is that users typing some content and post it. That's outcome we want. There's no friction to be removed at that step. I know you may think that some of the ideas may sound oversimplified or a naive, but remember that we're in the brainstorming face. The goal here is to come with some ideas, no matter how practical it is and how much business value it can generate.
另一种情况是用户输入一些内容并发布它。这就是我们想要的结果。在那一步没有摩擦力需要消除。我知道你可能认为有些想法听起来过于简单或幼稚，但请记住，我们是在头脑风暴的面孔。这里的目标是带着一些想法来，不管它有多实用，能产生多少商业价值。
Another way to come up with product improvement ideas is to seven users into different groups based on their behavior, and then analyze what makes users more active than others. How can we incentive eyes in active users to be more active on the platform? Let's use the same facebook interview question to illustrate this .. There are a few distinctive groups of users. We can analyze 1 group is the users who never posted. We want to figure out why to identify kidneys of those users.
另一种提出产品改进想法的方法是，根据用户的行为将七个用户分成不同的组，然后分析是什么让用户比其他人更活跃。我们如何激励活跃用户在平台上更加活跃？让我们用同样的Facebook面试问题来说明这一点。有几个与众不同的用户群体。我们可以分析出1组是从未发帖的用户。我们想弄清楚为什么要识别这些使用者的肾脏。
For example, we can figure out if there are new users, so they are not aware of this feature, or they don't know how to use it. If so, we could raise their awareness by increasing the size of the ui component, and providing more detailed instructions on how to use the feature. If they are existing users and they never posted, we want to find why that is happening. Do they find it difficult to create content? Or they don't have enough friends? So they don't have motivation to share. A simple way to gather feedback is to send surveys to the users. We could then address the needs for specific kinds of user. For example, if they find it difficult to create content, we could pregenerate some content for them.
例如，我们可以弄清楚是否有新用户，所以他们不知道这个功能，或者他们不知道如何使用它。如果是这样，我们可以通过增加UI组件的大小来提高他们的意识，并提供有关如何使用该功能的更详细说明。如果他们是现有用户，并且他们从未发布过，我们想知道为什么会发生这种情况。他们觉得创建内容很困难吗？或者他们没有足够的朋友？所以他们没有动力去分享。收集反馈的一种简单方法是向用户发送调查问卷。然后，我们可以满足特定类型用户的需求。例如，如果他们发现创建内容很困难，我们可以为他们预生成一些内容。
Another second of users is those who haven't posted in a while. They have posted before, but the post less frequently or they stop posing at a certain time. For this group of users, we definitely want to know why they become less engaged on the platform. Is it because they didn't get lots of attention such as likes and comments on their previous posts? Therefore, they don't have much motivation to post anymore. It is also possible that they may get some unexpected criticism on their posts, which make them feel it's a toxic platform. Once we can pick up the reason we could come up with ideas to address different needs for different people. If we find that people stop sharing was due to lack of attention to their previous posts, we could send some reminders and tell them that your friends miss you and they want to know how you are doing that could encourage them to start posting. If the reason is about criticism on previous polls, the platform could take more actions to remove negative comments.
另一秒钟的用户是那些有一段时间没有发帖的人。他们以前发过帖子，但发得不那么频繁，或者他们在某个时候停止摆姿势。对于这群用户，我们肯定想知道他们为什么在平台上变得不那么投入了。是不是因为他们之前的帖子没有得到很多关注，比如点赞和评论？因此，他们没有太多的动力来发布了。还有一种可能是，他们的帖子可能会受到一些意想不到的批评，让他们觉得这是一个有毒的平台。一旦我们能找到原因，我们就能想出办法来满足不同人的不同需求。如果我们发现人们停止分享是因为对他们以前的帖子缺乏关注，我们可以发送一些提醒，告诉他们你的朋友想念你，他们想知道你过得怎么样，这可以鼓励他们开始发帖。如果原因是对之前民调的批评，该平台可以采取更多行动删除负面评论。
The third group of users have the intention to pose and end up abandoned pose, where to talk about a few ways to reduce friction for those users. Here we can go one step further to figure out why we could ask them about their concerns. Maybe they worry that their post will not be popular or they think their content is too controversial. If we can get the reason we can have solutions to deal with them. If they worry that their content is too controversial, we could send some polls that's more controversial and encourage them to share their opinions.
第三类用户有心摆拍而最终放弃摆拍，在这里为那些用户谈几个减少摩擦的方法。在这里，我们可以进一步了解为什么我们可以询问他们的顾虑。也许他们担心他们的帖子不会受欢迎，或者他们认为他们的内容太具争议性。如果我们能找到原因，我们就能找到解决办法。如果他们担心他们的内容太具争议性，我们可以发送一些更具争议性的民意调查，并鼓励他们分享自己的观点。
The last group of users is those who posed frequently. We want to study them, right? We want to know what makes them more active and engaged than others. We can compare them versus those who are inactive. For example, we could use emotionally model and pick a few variables that you think would matter to change the number of post. Typically, we can consider a combination of user characteristics and their browsing behaviors as features, and use a tree based model to obtain the importance of those features. If we find out users with similar demographics are less active in a certain geographic area than others, we could then focus on that area and see if it's an awareness problem or if there are competitor products dominating the market. We could consider to launch a marketing campaign to raise awareness of those users. I have just share with you two ways to come up with ideas to improve a product.
最后一组用户是那些经常摆姿势的人。我们想研究它们，对吗？我们想知道是什么让他们比其他人更积极、更投入。我们可以将他们与那些不活跃的人进行比较。例如，我们可以使用情感模型，并选择一些您认为重要的变量来改变帖子的数量。通常，我们可以将用户特征及其浏览行为的组合视为特征，并使用基于树的模型来获得这些特征的重要性。如果我们发现具有相似人口统计数据的用户在某个地理区域比其他用户更不活跃，那么我们就可以关注该区域，看看这是否是一个意识问题，或者是否有竞争对手的产品主导市场。我们可以考虑发起一场营销活动，以提高这些用户的意识。我刚刚与您分享了两种改进产品的方法。
During interview. There's no need to provide both approaches. You can only focus on one of them. My wrong son is that if my interviewer is a product manager, i'm gonna go for the first option. Basically, i'm gonna try to reduce or remove any friction in current user experience. I won't mention anything related to emotion learning. If my interviewer has data science background, i'm gonna go for the second option. Basically, i'm gonna analyze you the behavior and see how we can turn in active users into active users. After the brain compensation.
在面试期间。没有必要同时提供这两种方法。你只能关注其中一个。我的错误儿子是，如果我的面试官是产品经理，我会选择第一个选项。基本上，我将尝试减少或消除当前用户体验中的任何摩擦。我不会提到任何与情绪学习有关的事情。如果我的面试官有数据科学背景，我会选择第二个选项。基本上，我会分析你的行为，看看我们如何将活跃用户转化为活跃用户。在大脑补偿之后。
The next thing is to make recommendations on what ideas to prioritize. If you have mobile ideas, there might not be enough resources in reality to support all of them. So we want to recommend how to select the idea to focus on. We can recommend doing some quantitative analysis, analyzing the proportion of users that are impacted by each idea and compare the business impact. Back to the interview question from facebook, we can get the percentage of users who never posted users who haven't posted in a while, and users who have the intention to post, but ended up not posing anything. Once we have those numbers, we can select ideas that may impact the most users. Another way to prioritize idea is to recommend those that are most cost effective, mean that we could put less effort to drive more impact. For example, we can focus on the users who have the intention to post, but end up not posting anything, because there is just one step away from posing, assuming that there are few amount of users who are in this category. We want to prioritize ideas that can encourage those users to post.
下一件事是就优先考虑哪些想法提出建议。如果你有移动的想法，现实中可能没有足够的资源来支持所有这些想法。所以我们想推荐如何选择要关注的想法。我们可以建议做一些定量分析，分析受每个想法影响的用户比例，并比较业务影响。回到Facebook的面试问题，我们可以得到从未发布帖子的用户的百分比，有一段时间没有发布帖子的用户的百分比，以及有发布意图，但最终没有发布任何内容的用户。一旦我们有了这些数字，我们就可以选择可能影响大多数用户的想法。另一种优先考虑想法方法是推荐那些最具成本效益的想法，这意味着我们可以投入更少的精力来推动更大的影响。例如，我们可以关注那些有发布意图，但最终没有发布任何内容的用户，因为距离发布只有一步之遥，假设属于这一类别的用户数量很少。我们希望优先考虑能够鼓励这些用户发布的想法。
The next two steps can be optional, depending on the type of the interview. If it's technical phone screen, it might not be necessary, but it is on site interview. Most likely, the interviewer will ask you some follow up questions, such as how to design an experiment to test if it's a good idea or not. The third step is to define one or two success metrics to measure the success of the new feature you proposed. This is very similar to the major success type of question we've covered in another video. This step is particularly necessary when you ask to that into how you would design an experiment to test your idea, be careful about how we split users and whether there will be interference between control and human groups, because that will make the results unreliable, especially in social networks, such as facebook, linking and twitter and two side markets, such as air bnb lived and uber.
接下来的两个步骤是可选的，具体取决于面试的类型。如果是技术电话筛选，可能没有必要，但这是现场面试。最有可能的是，面试官会问你一些后续问题，比如如何设计一个实验来测试它是否是一个好主意。第三步是定义一个或两个成功指标，以衡量您提议的新功能是否成功。这与我们在另一个视频中提到的主要成功类型的问题非常相似。这一步是特别必要的，当你问到你将如何设计一个实验来测试你的想法时，要小心我们如何划分用户，以及控制和人类群体之间是否会有干扰，因为这将使结果不可靠，特别是在社交网络中，如Facebook。链接和Twitter以及双边市场，如AirBnB Live和Uber。
Check out another video on creaking advertising problems to learn more about the pitfalls and solutions for every time. The last step is to summarize the overall approach of taking to improve the product. We want to summarize what the goal is and how we come up with a few ideas to improve the product. How do we prioritize the ideas based on the business impact? And if we want to run evidence, what metrics do we use? And how do we design the experiment? So there you go. I have just shared a few steps that I typically follow to answer this type of question. If you have different ideas or different ways to approach them, I'd love to hear them, feel Free to comment alone. As always, I appreciate you for taking time to watch this video. And any feedback common question is welcome.
查看另一个关于吱吱作响的广告问题的视频，了解更多关于每一次的陷阱和解决方案。最后一步是总结改进产品的总体方法。我们想总结一下目标是什么，以及我们如何提出一些改进产品的想法。我们如何根据业务影响对创意进行优先排序？如果我们想要运行证据，我们使用什么指标？我们如何设计实验？所以你去那里。我刚刚分享了几个我通常用来回答这类问题的步骤。如果你有不同的想法或不同的方法，我很乐意听听，请随意单独评论。像往常一样，我感谢你花时间观看这个视频。欢迎就常见问题提供任何反馈。