Billo.app Case Study | Vytas Ramanauskas

/ Order experience & acquisition

The problem & the opportunity

Problem/opportunity from the venue/customer/employee point of view?

The problem & the opportunity lies both from business' and customer's side:

Billo as a business would like to acquire & convert more customers (and to increase the revenue so the business becomes profitable).

Customers (brands) would like to have better experience while making an order and to be able to choose from more offerings.

Opportunity also lies in the fact that e-commerce market is expected to be around $5.55 trillion in 2022, and the number of online retailers is growing every year.

User-generated content (UGC) market size was ~$3.58B in 2021; it's projected to grow to ~$18.65B by 2028.

Billo goal is that every 10th UGC video ordered in the world would be through Billo platform.

Why is this important to the business?

Billo is still looking for a product market fit and has to increase its offerings to become profitable. Revenue and net profit growth are the main company level objectives at the moment.

Success KPI from a product perspective

purchase conversion rate to grow by 50% (currently is ~2.8%)
to lower the video delivery time: 21-28 days delivery segment should not be more than 5% (currently - 14.14%)
with an increased number of customers ( and orders), to grow the retention (1 month) to 30.0%
to keep customer satisfaction at least 80.0%

Recurring appts.

Gathering feedback from customers

We already had quite a lot of user feedback on the Billo platform being:

hard to use,
not user-friendly,
having poor support and
long delivery times.

We have gathered some additional feedback using post-delivery survey (via Typeform) and interviewing our customers (brands):

As mentioned earlier, there were some common topics for the pain-points our customers had - some were pretty clear and others were abstract.

Clear improvements:

video delivery speed
no direct communication between the brand and the creator
no creator profile or historical data
ability to save task (order) as a draft

Abstract ones:

poor or slow customer support
ability to add more materials for the creators
usability problems

Me, together with the Product manager, put the ideas and pain-points into an OST (Opportunity solution tree) which gave a good perspective of what we can tackle first.

The prioritized issues were converted into OKRs and initiatives (see above).

Platform areas to tackle:

onboarding and first order experience
managing the order
retention

Initiatives for onboarding and first order experience

optimized brand category values
digital products introduced
new physical video types introduced
product & brand pronunciation helper
duration-based pricing
stop selling subscriptions
saving task as a draft

scenario & mentions re-redesign
scenes & mentions warning
add-ons selection re-design
display summary section
auto-save tasks (orders)
additional details field

video delivery time improvements - shipping tracking introduced

display all-rounded creator's profile
creator tags
don't allow new creators to apply to first time customer tasks
adding creators to task

reducing zero-moments
a bunch of CRO-related improvements (web + mobile)

resurrecting drop-offs (now)
creator showcase to invite when creating the task (now)
UGC Photos add-on (now)

Initiatives for managing the order

weekly email notifications + notifications
content auto-approval
aspect ratio & duration checks from the creator's side
video cards re-design
tracking product shipping
limit edit requests per task

direct communication between brands and creators

Initiatives for retention

video ratings
template gallery
invite previous creators to the task
inviting users to collaborate on brand
high value customer growth
pinning creator to task

At first, the biggest assumption was that long video delivery time prevents brands from placing the orders and having great experience with Billo.

Since we started tracking, the average duration was ~18.4 days for a brand to receive a video.

It seemed way too long and the user research (surveys & interviews) confirmed that.

Tackling the video delivery time

Shipping tracking

Video delivery time consists of these stages:

(brand creates a task)
creators apply to task (24 hours)
brand chooses the creators they like (~2.1 days)
brand sends products to each of the creators
creators receive products and shoot videos
creators upload videos to Billo
brand reviews & approves video
(might be some edit requests - then creator has to re-do the video and upload to Billo again)

Creators usually apply within the first 24 hours.

However, there was no notification for the brands to go do their part - to approve the creators. Time to approve really differed - spanning from 12 hours to almost 4 days:

The easy fix was to send brands an email reminder to go approve the creators they like:

However, this did not really make a difference. The average time for brands to approve creators to task is still around ~2.1 days.

We also added a reminder that the faster brands ship their products to the creators, the faster they will receive videos - it gave some more clarity about the process, too:

However, the longest part was from when the brands sent the product until the creators uploaded the video.

The deadline for the creators was 3 days after they receive the product.

Creator interviews (8) made clear that the creators usually plan when they are going to shoot the videos, meaning that if they receive the product on Monday, usually they still shoot the video during the weekend.

Most of the creators have day jobs and are pretty busy. They tend to shoot multiple videos at once (one after another).

It was clear that the creators don't really comply with the intended deadline (3 days), however we couldn't blame them if we didn't know when they precisely receive the products.

We were in the dark when it comes to reliably understanding whether the creator can already start working on the task as we don't know whether they've received a product. If we would know that, we could:

communicate the task progress to the brand - and set the right expectations
encourage the creator to deliver the video faster
not cause frustration with creators blaming them for slow delivery when it's not their fault.

This would also allow to track delivery progress for around 80% of shipments + receive real-time updates on shipping events.

Target audience

Everyone (brands and creators) involved in tasks where a physical product needs to be delivered to a creator prior to video production. That’s currently roughly 90% of all tasks.

Customer insights

Task delivery duration is in the TOP-3 concerns by the brands when working with Billo.

Success metrics

80% of all newly created product shipments should be trackable
creator avg. video submission time should go down by 1-2 days (currently ~5 days)

Now, brand needs to enter the shipping tracking number and we fetch the shipment (using trackingmore.com API) - then we can display the statuses and the shipping service company.

From the creators' side - we started to display the notifications about product shipping status - and when the tracking company displays that the product was delivered - we start to ping the creator and ask whether he received it.

Finally, after some additional brand interviews - it became clear that brands do care about the video delivery, however a reasonable amount of time is fine for them.

Meaning, that they don't expect to receive a video in 5 or 7 days - they are fine with up to 14 days.

Since we didn't want to threaten our creators community and impose some sort of sanctions if videos are not uploaded on time, we thought of changing our goal to:

videos that are being delivered in more than 20 days are no more than 5%

This would ensure that the expectations from the brands are met.

From the creators side - we wanted to ask for a realistic behavior - so we increased the deadline for the video upload from 3 to 5 days.

Additionally, to make them feel it's worth it to meet this deadline - we introduced creator's statistics.

This would serve the both sides - brands already asked for a better way to choose the creators.

The initial idea was to display these stats about the creators:

# of videos created
rating
reliability score

At the same time - we also displayed these stats to the creators as well. It boosted their motivation to keep their scores high so they have a better chance to get picked.

For the MVP - the on-time delivery algorithm counted:

shows % of videos uploaded within 5 days.
only videos in the last 365 days and from 3 to last 20 videos are calculated
only first upload is counted, edit requests not included in MVP algorithm.
reliability score re-calculated after slot approved/unapproved ant slot auto approved

The result proven to be worth it - the all time avg. video upload dropped from ~5 days to 2.85 days (last 30 days)

this is from 'product received' action until the video being uploaded
we still feel the creators are cheating a little bit since the shipping tracking suggests that the products are delivered earlier - so there's a discrepancy for ~2 days
the overall trend is going down

Looking at the time intervals:

for videos being delivered in 21+ days the trend is dropping - from 14.03% (last 12 months) to 6.42% in the last 30 days
28 days - 3 month delivery - dropped to 0%

First order experiene

Optimized brands' category and type values

To improve the first order experience, we delivered a bunch of other initiatives, too.

First, we optimized the brand category values - since there were quite a lot of brands being not categorized or being under the 'Other' category:

Removed: Restaurants, Telecommunications, Events, Banking;
Restaurants → Food & beverages;
Telecommunications → Services
Banking → Services

We have also migrated some other previous values:

Distributor → Physical
Entertainment → Digital
Manufacturing → Physical
Retail → Physical
Services → Digital
Other → Physical

For the brand type - we only left 3:

Physical product (most eCom)
Digital product (App/Game/Service/Website)
Agency (representing other brands)

Digital products introduced

With this optimization, we also started to update the Digital products category - we were being asked by our Brands to establish a convenient process for videos without physical products.

What's the opportunity?

Positive impact to marketplace liquidity (not needing shipping improved task delivery time), also - it would be a new acquisition channel.

Introducing Digital products would be a way to a better PMF (Product market fit) as virtual product (online service, website, online game, app, etc.) is quite common type of product worldwide:
- 831 Brands (Apps type) in the database / 206 Brands (Apps type) ordered during 6 months;
- 136 Other, Services, Online store (Brands types) of Software category in the database;
Introducing Digital products would result in bigger LTV (Lifetime value) as Client could make more conversions with shorter delivery time;

Target audience:

Clients, who have digital products/service;
Creators in English speaking markets;

Success metric:

to improve the avg. video delivery time for digital products;

To support digital products, we needed to adjust our 'Add a new product' functionality first:

The Brand now can mark product as digital → so the video delivery deadline is also shorter by 4 days.

For the creator, we started to display product access instructions (usually a link).

Billo Marketing team started to promote our services to digital brands - and nevertheless the numbers for digital products were not that high at the beginning, now we see around 15% of all products being created as digital.

New video types for digital products

At first, Billo only offered the same video types both for physical and digital products.

Another experiment was to continue to explore the digital products area, so we introduced 5 new video types:

Video ad
Promo
Testimonial
Explainer
Preview

Video ad - a flexible ad for any digital product or a service;

Promo - reveal benefits of your app with an over-the-shoulder device shot;

Testimonial - benefits of your app in a split-screen video or from a point of view;

Explainer - run-through your app to explain how it works and what it does;

Preview - intro video for the app store listing.

From product perspective, it was not much to do - just adding some new offerings to Billo platform.

It's a miss, though, since these new digital product video types are among the least popular (all in all just a few %), and we kind of stopped experimenting with them.

New video types for physical products

Since the business strategy is to increase our offering - we also added some new video types for the physical products:

Testimonial video
How-to video
Unboxing video
360 video

With introduction of those, we decided to experiment with offering a higher quality videos - so-called 'Premium type'.

What's a Premium type quality video?

Basic quality is a home environment video. It’s simpler and usually shot on a smartphone.

Premium quality videos are produced by a selection of more experienced creators with home studios. Expect more attention to detail: better lighting, solid color backdrops.

From the creators' side - our Supply team prepared the materials and guidelines on how to shoot these new video types, and inside the App added support for it.

301 - Task details - Context info - 6.png

With that being said - the new video types were an instant success - respectively Testimonial being around 20% of all video orders, and How-to being 6.5% (unboxing is around 3.5% and 360 - 0.76%).

Broader offering definitely helps to serve more use cases that brands have.

Duration-based pricing + no subscriptions

Among the new video types being introduced, another big bet was to start selling videos with a duration-based pricing.

That also meant dealing with our support team overload by not selling subscription packs anymore. These were tricky right from the beginning and a lot of brands were asking for extensions, or for a refund if they were not happy with the video quality.

Old pricing:

Another problem with that was that our marketplace was not really healthy based on what we pay our creators - they were avoiding taking longer video tasks since the payout was still the same - $20/per video.

With the duration-based pricing we had an opportunity to do right by them, and at the same time take more money from the brands (resulting in higher GSV).

The new pricing split was introduced:

15s - starting from $59 (payout - $20)
30s - starting from $79 (payout - $30)
60s - starting from $99 (payout - $40)

'Starting from' means the basic video quality; premium quality videos are more expensive, and this split also allowed brands a bigger freedom of choice.

Currently, 30 seconds video is the most popular (60.37%), 15 seconds being second (24.9%), and 60 seconds being third (14.73%).

Premium quality grew to 28.63% (vs basic - 71.37%).

Overall, these pricing (and video quality) changes proved to be a success - the average order value grew significantly (from ~$128/order to ~$164/order).

The creators are happy with a more fair compensation for their work - it makes the marketplace healthier.

Brands, obviously, are not happy in general for the raised prices, however, now they have way more freedom to choose the type of video they need.

Small purchase funnel improvements

From doing usability testing (39) (remote, unmoderated), I verified my assumption that Billo is still unclear, not too user-friendly, and even chaotic, when it comes to acquisition of new customers.

Most of them expressed that they were "overwhelmed", "very confused", also said that the platform is "poorly organized".

I wrote the testing scenario and tasks - which included perception questions (such as 'what is the first thing you notice? ' or 'what do you think is your next step?'), and usability questions - whether they can finish an order.

After registration, they landed to the 'New task create' screen which, essentially, is a huge form with a lot of choices.

Splitting the steps

The question is who cannot get lost in this screen?

Some bigger changes are still in-progress, but the main idea I have is to split the task creation flow into a few smaller steps.

For example:

Step 1 - add a product
Step 2 - video options
Step 3 - video specs
Step 4 - add-ons
Step 5 - checkout
Step 6 - additional details

Scenes & Mentions

Sunk cost is one of the reasons, and the other is what I'm constantly hearing from the user interviews and usability testing - the flow is not intuitive, the process is not clear.

Currently, we split the flow into 2 (and soon to be 3) steps:

Step 1 - the whole form
Step 2 - checkout
Step 3 - additional details

To simplify placing the first order, we placed 'Scenes & Mentions' into the 3rd step.

Scenes & mentions is one of the most tricky part for the brands - since they need to know what they would like to see in the video.

We removed the customization with Scenario A and Scenario B - the whole thing became simpler.

Also, we added some good suggestions as the placeholders. If the brand adds too many scenes for their selected video duration - we display a warning, too.

The checkout was transformed from a modal window to a page (step 2).

Small UX improvements

Apart from split steps, we also did a bunch of small UX improvements - from introducing auto-save, to not deleting the scenes & mentions after navigating further.

Some other mentionable ones:

introduced the save task functionality
country pre-fill when entering credit card details
country selector has a US as a default
now we send an email reminder to finish entering the task details

Creative mode

For the scenes & mentions, we also experimented with 'no-scenario tasks' or so-called 'creative mode', where if brand toggles this selection - it's up to the creator to think of the scenario.

Product fetch

Looking at the purchase journey, a huge drop happens between the first and second step:

Adding a new product sometimes seems to be impossible - usability tests show the same - a majority of testers do not add their product and they try to proceed with the task details without it.

To tackle this problem, we decided to fetch the product details automatically. The step is being simplified since we fetch the product details from an active url being pasted.

Screen Recording 2022-10-25 at 22.33.55.gif

Brand attributes fetch

The same service is now being used to fetch the brand's logo and description. Brand enters this data and their website during the sign up, and around the time sign up is completed, we already display their brand attributes.

This takes off a huge amount of work load from our support team, who was constantly searching and uploading these brand assets - since we display them to our creators. At the same time - it makes brand's life easier.

Onboarding screen

Finally, after analyzing the usability tests, it became clear that perception-wise our users are lost.

They are wrong when answering question what is their next step, and also they are drawn to the wrong elements on the screen, when they land after signing up.

I designed additional onboarding screen to experiment whether it can make a difference for our customers' perception.

Main goals were:

to guide their focus to the next step
to explain how Billo works (the process)
make inspiration videos easily accessible

We have only released it a few weeks ago, however, we can clearly see an improvement from our testers (perception-wise).

Now they are saying, that in that particular step, they are confident, they know what their next step is, and they also notice the 'How Billo works' video.

Conversion-wise it looks like this screen does not really makes any difference (good or bad), but if our customers can be less overwhelmed after signing up, I think it we can count it as a good improvement.

This pretty much sums up the purchase funnel improvements.

Outcome is way more important than the output. With that being said - our main goal was to increase the purchase conversion rate by 50%.

Started from an avg. of 2.8%, the current week-by-week fluctuates between 3.5% to 4.68%. Since the last updates were made a few weeks ago, it is hard to know whether these numbers are statistically significant yet.

However, looking from the 12 months perspective, the trend is definitely going up:

Adding more creators to task after order

One initiative needs some additional attention:

Adding more creators after the order is placed delivers around $25K additional revenue every month.

The hypothesis

Customers would be willing to add additional creators to task if we gave them an easy way to do so.

Key result

Additional videos booked by adding creators to task - at least $25K/month.

Our support is currently manually adding ~50 creators across ~20 tasks per month. That’s ~$5K GSV per month. By building the motivating flow we should be able grow at least 5x this number.

Even if only 5% would opt in to adding additional creators, we end up generating ~25K additional revenue per month

Currently, after creators apply to a video task and the brand approves them - the brand has to start the shipping process.

Instead of going directly to shipping screen, we thought it would make sense to show the rest of creators who applied to a task but were not chosen. If the brand ordered 3 videos and they liked 5 creators, there was no way to add 2 more to your task.

The solution was to display the creators who left out - even after all bought creators were approved to task.

Now, the brand can select as many more creators they like and to update their order by paying for it.

After that, the next step is the same - shipping products to all of the creators.

The end result

On average, we make ~$20K per month only from this functionality (from around 77 additional creators selected every week).

Managing the order

For order management, there were a few initiatives worth-mentioning, too:

weekly email notifications + notifications
content auto-approval
aspect ratio & duration checks from the creator's side
video cards re-design
tracking product shipping
limit edit requests per task

Email notifications

Optimizing our email notifications was a low-hanging fruit. There were a bunch of emails being sent every day - for every task ordered.

We aggregated them into a one daily email - with all the action items only. If there's no action - there's no email, only a weekly summary of what's happening.

If the final video is ready - we notify customers instantly now.

Content auto-approval

If the brand doesn't approve the video within 3 days - we auto-approve it automatically.

This helps to create a payout to our creator, and do not leave him hanging. Also, if the final video version was uploaded and no edits were requested - the video should be approved as final.

Creator's duration & aspect ratio checks

To ensure there are no edit requests related to a wrong aspect ratio or video duration, we implemented in-app validations:

Also, we started to limit edit requests per task - maximum of 2 edits - since the task cannot stay in-progress for ever. This would also raise the frustration of our creators.

Video cards re-design

We had a numerous reasons for improving the video cards - the main one being that the information we'd like to display for the customers doesn't fit.

Also, the video player was too small as our customers mentioned. The video is the most important results after our services are done, so it has to be big enough to view it and decide whether it's good enough or not.

Some of the early version cards did not have the creator statistics displayed and even the avatar.

Also, the reject option was hidden pretty deep which caused quite a lot of pain.

Finally, we displayed 4 different aspect ratios, and some of them was too small - hard to comprehend what's happening there.

The new video card is consistent size, always displaying a full-sized thumbnail.

Also, there are only two main action buttons, and we added a creator's text pitch - another personalized piece to help creators pitch themselves to tasks.

Clicking 'play' icon opens a modal where we have plenty of space to display more information about the creator:

we introduced the bio field
creator tags now available and indicate what the creator is into or what can he show
most importantly - we now display the creator's video pitch and other videos that represent him (story-type of interaction).

This is still being tested out, however we already heard some positive feedback from the brands saying that we made it easier to choose between the creators for their tasks.

Retention

Retention is always an important topic, and we have plenty of ideas on how to improve our retention after 1 month.

In Billo's case, the retention drops every single month, so we're trying to catch our customers after a few weeks to place a new order.

With the goal of keeping 30.0% retention after 1st month, we tried to build better relationships between the creator and the brand, and also between the brand and Billo.

Video ratings

All the major successful marketplaces have ratings - be it AirBnb or other businesses.

Services such as Uber also asks to rate the journey after the service was provided.

Our thinking was that ratings can be both a great way to gather some insights whether our customers are satisfied with the final result, and also to build some new touch-points with the brands.

The very first iteration was simple - just asking whether a brand liked the video or not. If not - we ask to select what they didn't like, and provide an open-text comment field.

It was a great start - now we were getting real feedback:

However, the thumbs up/down rating seems not to be enough - we needed more granular feedback so it can be actionable.

Another reason - we were planning to display ratings to brands, so they can choose the creators accordingly.

For this reason - we updated the ratings steps into more granular ones:

This, finally, gave us a real picture of how customers like our services:

Currently, the average video rating is 4.87, with 91.35% of ratings being 5-star.

Since August 2022, the lowest ratings (1/5) have dropped to a minimum:

Finally, the comments give us even more context of how we are doing:

Performance ratings

On the final iteration we introduced performance rating step. This is closely related to retention - since we launched a promo code (15% discount) for those who rate video's performance.

This has been released this week only, so there's no data on the performance.

Template gallery

An initiative that wasn't a pure success case, from my point of view.

Template gallery tackles customers' objection - they are not sure whether Billo can provide a video that addresses their needs.

Do we have relevant creators? Is the video quality good enough?

Also, we were looking at employing current content that’s been produced on Billo to help and inspire new and existing customers for deeper engagement and habit building.

The idea is to build a content loop:

Find out what the best produced content is
Showcase that content to inspire new task creation
Facilitate task creation by reusing previous task inputs (brief) and outputs (final video)
As an experiment to validate the approach, we’ll first launch a static content page within the platform and measure engagement with it.

We started with a few brand categories - and hand-picked videos that represents those categories the best.

Videos are both basic and premium, with add-ons and without - so the customer have the freedom of choice.

Brands can see the video and it's specifications, see what was used, and most importantly - they can use it as a template for their order, meaning that all the options are pre-selected for them.

Target audience to test with

We launched the experiment with a relatively well understood segment: Cosmetics. It accounts for around 15% of all rated tasks so should provide good insights on user behaviour.

It took 3 weeks to fully implement but the first numbers were not optimistic.

Our Support team was noticing that first time customers tend to struggle with defining a good brief. Data proves that: edits are requested for 8.5% new user tasks and only 3.1% of non-first time user tasks (we may want to revisit this data as it may not be too reliable).

So the template gallery should tackle that - since we pre-fill their briefs.

Success metrics

Overall conversion of the tested segment vs others
Purchases inspired by the Template Gallery

Summary and lesson learned

This pretty much sums up the purchase funnel improvements.

Outcome is way more important than the output. With that being said - our main goal was to increase the purchase conversion rate by 50%.

However, looking from the 12 months perspective, the trend is definitely going up:

Success KPI from a product perspective

purchase conversion rate to grow by 50% (currently is ~2.8%)
to lower the video delivery time: 21-28 days delivery segment should not be more than 5% (currently - 14.14%)
with an increased number of customers ( and orders), to grow the retention (1 month) to 30.0%
to keep customer satisfaction at least 80.0%

Another big task was to lower the video delivery time - and ensure that we don't have a lot of videos being delivered longer than 21 days.

The goal was making sure the number is less than 5% of all videos delivered.

Started at 14.14%, for the last 30 days we have 6.42% - so the goal is almost reached.

#3 product team goal was to keep the retention at 30.0% with the increased number of customers (1 month).

The numbers are looking optimistic:

And finally, Billo GSV (gross sales value) is consistently growing with upward trend - we had a few record months after months.

This month (as of October 2022), we're about to surpass half a million in monthly revenue - 1st time in company's history.