Folksy Ltd

Questions for JamesB about testing


(James Boardwell) #1

Hi,

I said I would do a chat today and even though the AB test is still running on the Gift Guide promo - so we don’t have a winner on the experiment, I’m happy to field general questions about development work and testing.

Leave your questions for me here until 14:00 and I’ll respond to them at 15:00. I’ll spend 30 mins or so and try and get through as many as possible.

Thanks!

James.


(Kim Blythe) #2

How long will the test be running for?
The ‘Big button’ has been there for over 1 week now…there must be some indication of how it has affected views and sales.
I know my sales have just stopped. I did well in November, but only 1 sale since the appearance of the gift guide button, and that was to another Folksy seller.


(James Boardwell) #3

Hi Kim,

Google determines the length of the experiment and so far it states “Experiment is still running - no winner can be determined yet”. Google’s algorithm for determining success is quite complex but is ultimately based on the revenue generated through the different routes (original and promo) - more here:
https://support.google.com/analytics/answer/6112437?authuser=2

We have run experiments lasting 4-5 days and also some that ran for over a month before Google determined a winner.

In terms of sales, I can see that you have had an increase of 21% in views of your work in the last 7 days over the previous 7 days. Perhaps some of the items that have been successful in November are because they are for Xmas trees and I suspect that most Xmas decorations are now tailing off? It would take quite a bit of investigating to look at your traffic and see where it came from and what they were searching for etc. and even then it’s not possible to state with any confidence if the drop in sales is due to anything specific such as the gift guide promo but seeing as none of the gift guides specifically have Christmas Decorations like yours (which have sold recently) I’d doubt it.


(Marg) #4

The “Folksy finds” have quite a few items that are sold. This seems a waste of space, when another item from another seller could be used. Lately I spotted a couple of shops featured on the front page that were empty, which seems a wasted opportunity for someone else to be in that spot, and possibly a bad experience for a new shopper to Folksy.


(Sara Leigh Thornton) #5

Have you actually asked any customers what they think about having the link there? Those I have spoken to don;t like the extra fussiness on the page, and if it has to stay would prefer it to open the link as another page so they don’t lose track of the shop they were in.

I don’t understand all this ‘bounce rate’ stuff, but if the button was put there because people weren’t exploring Folksy after visiting one shop have you taken into account that many of us will direct people to specific items in our shop from other places, and these people often want to just quickly buy that item, They might come back and browse Folksy another time after that, but I know with things like my pet signs and pet baubles I direct people here from Facebook and a Forum as they can just look at my listing and get the full specifications of the item. It’s unlikely that on these occasions they will go on to browse, regardless of whether there is a link to the gift guides or not.

It would be nice to have a wider range of gift guides. There is lots of art on here, but it never gets a gift guide of its own. If the gift guides were easier to find from the front page (I can never find the flipping things when I’m looking for them!) then perhaps we wouldn’t need the button?


(Mairi A) #6

Hi
I’m wondering if you would be looking to set up focus groups in the future for sellers to be involved in improvements to the site? If so, count me in.

In terms of development, as an individual seller I’ve found that my views/sales are right down on folksy. I’ve done just about the same level of promotion so I’d like to see a better emphasis on keeping my potential customers within my shop.

  1. by having bigger pictures of my other items in the listings, and
  2. moving the links that highlight the categories I’ve listed the product in out of the area that is sandwiched between the ‘designed by…’ and my shop details/icon on the right hand side of the listing. I feel that placing them between two parts of info specific to me would make a customer think that is where they would find that particular category of my shop- and it takes them somewhere else instead.

Lastly - I’m neither up nor down over the gift guide icon as I never noticed it, but thank you for listening to the concerns of sellers when it was raised .


(Witty Dawn) #7

As others are asking "when is this test ending’ ? saying 'it’s still running ’ is not that helpful a reply to a very specific question. Neither is giving a broad generalisation regarding how long other tests have run…

Many thanks


(Witty Dawn) #8

I have referenced focus groups in virtually every communication regarding ‘The button’ and I have yet to recieve a comment / specific reply regarding it. It’s been mentioned before wrt other implementations in the past as well.


(James Boardwell) #9

Hi Marg,

Yes, we include sold items in the Finds because we believe it helps to add a sense of jeopardy to the buying process. If people know things are sold and selling then they are more likely to buy. We trialled the view and it was marginally better performing with sold items included.

Empty shops - yes, we tend to exclude those and they should only appear if they have recently sold out.

Hope that helps,

JB


(Kim Blythe) #10

Yes, My views have gone up recently, but my sales have gone down…so obviously they are leaving my page without buying…and I am bound to think that this is because of the Gift guide button luring them away…bearing in mind my good sales prior to the button…


(Hazel Rayfield) #11

James do you have any plans to promote Folksy externally as well as internally?

I would be interested to hear any plans for 2015?

My shop has not done so well recently, coinciding with when I took out the plus account for some unknown reason. I was doing ok here so went for the plus account but my sales have dropped right off. I would be worried but my shops elsewhere are going well :slight_smile:

I do agree with Sara it would be good to see an Art based guide.


(James Boardwell) #12

Hi Sarah,

  1. You mean the gift guides promo link on the product page? We did not ask prior to the test and with hindsight we probably would have done.

  2. Yes, we do understand that many people come to product pages from social channels and - as mentioned in the blog post - a significant number (50%ish) come via Google search too. So, you may be right in that many don’t want to browse further if they have been directed specifically but this is probably a minority and it would seem odd not to offer them choice anyway. But we will leave the test to work out if it works. It may not. The promo may not be the best way to support browsing, but we’ll continue to support buyers in whatever way we can to find things they want to buy.

  3. A wider range of gift guides. Yes, it would. We theme gift guides and art will be featured if it fits a theme - I can see quite a lot of artwork and surface pattern work in the existing guides but we will continue to make sure it is represented. And it would also be good to find them and we’re working hard on that. Most people don’t come to Folksy via the front page but I do understand it is a key way in for many returning visitors.

Hope this helps,

James.


(James Boardwell) #13

Hi Kim,

As I said, without looking at the detail (and you could do this if you use Google Analytics - http://folksy.uservoice.com/knowledgebase/articles/57282-using-google-analytics ) it’s not possible to explain why (other plausible explanations are that the kinds of things you sell are decorations and they may be tailing off?). Fluctuations in sales is very common and can be due to a variety of reasons. It might seem like extra work but knowing your way around Google Analytics can really help you to understand more about your visitors and buyers and consequently help you make better decisions about what to make and how to promote it.

Best,

James.


(Hazel Rayfield) #14

Yes it is James and thank you my art appears in a couple of these, but maybe a section for original art could be considered as there is one for print and digital art I beleive.


(Catherine & Jennifer) #15

Sorry I’m late to this thread. I hope it’s OK if I add a question.

Are there any user testing results regarding the signed-in vs non-signed-in front pages yet? It’s been there a month now and I’d like to know what impact it’s had.

For what it’s worth, I find the signed-out version very dull and uninviting. On first impressions it’s not that obvious it’s a retail site and the categories are below the fold, something I’m not used to seeing on a retail site. With just three images, none of which are direct item links, it’s very limited and as a lot of my customers last year were guest buyers it concerns me that this is all they see on the front page.


(Christine Shephard) #16

James, can you explain why you feel the need every year, just before/during our busiest selling period, to start tinkering with, implementing and testing various parts of the site? There are many other times during the year, when it’s not as critical for us, that you could spend time doing all this without impacting as much on our businesses. I’ve been on here for 3 years now and I don’t think there’s been an uninterrupted pre-Christmas season yet. Every year you apologise and say the timing could have been better, and the following year you do exactly the same thing again.


(James Boardwell) #17

Hello Witty Dawn,

  1. Google Analytics experiments are not time specific unfortunately, they are based on data about conversion rates and measuring what performs best. We’re at the mercy of Google (but thankfully they are smart and their tests are very rigorous)!

  2. We have run focus groups in the past but we wouldn’t usually run them to help us design features and functions of this nature as they probably wouldn’t be appropriate - groups tend to be better used for broader, general understandings and behaviours. Individual interviews and user tests of wireframes would perhaps heave helped us gauge the reaction, but as we want to support buyers we would have mainly focused on them. However, as mentioned in a previous comment we would have communicated the promo work earlier to sellers to understand any issues and concerns and this was an oversight.

Hope this helps,

James.


(James Boardwell) #19

Hi Hazel,

I mentioned about paid for advertising in an earlier comment and so won’t repeat what I said there, but yes, we’ll be continuing to promote Folksy externally (we do a lot of it already) mainly through content marketing and partnerships.

I can see that your monthly average views bely a very variable weekly and daily views, but that in Jun, Sep and Oct you were below your average whereas in Jul and Nov you were above average. It’s harder for me to drill down into your conversion rate without quite a lot more work but it might be worth seeing if your sales correspond to any particular piece of marketing or promotion you are doing and try and define what is (or isn’t) working.

Thanks,

James.


(Kim Blythe) #20

You mention you want to support buyers…but I think supporting your sellers here is just as important…as Witty Dawn said, without us the buyers have nothing to buy…and I think most of the problem here is that we feel we are not being taken very seriously on this issue, and that sellers should be more involved before changes like this are implemented.
For some people on here this is their livelihood and if it doesn’t work for them they will go elsewhere.


(James Boardwell) #21

Hi Catherine and Jennifer (which one is this?!),

Good question. It’s hard to test like-for-like pages as they are both folksy.com - however if we look at new visitors to Folksy landing on / then we can see no significant change in conversion rate. So, on balance, considering the work we put in, it hasn’t been a success. The work came out of user research that showed new visitors found the logged in homepage (what was the homepage for everyone) overwhelming and didn’t know where to go / start their journey. This is still true and in user tests people do prefer it - but the data shows it isn’t any better than what we had so we will look to revisit how we support new visitors again.

It’s worth noting that the signed out page is not for you. In fact I’d expect you to find it dull and uninviting as you’re used to lots of content and change, but people new to Folksy seem to want simplicity. One to look at again!