topic Re: Reviews AB test in General Zazzle Discussion

Reviews AB test

Scott — Wed, 18 Sep 2024 22:07:05 GMT

Hi Zazzle Creators,

We want to inform you about an upcoming A/B test aimed at improving the product review process. These changes result from thorough research and are designed to enhance customer trust and overall experience on the Zazzle marketplace.

Our commitment to transparency and integrity remains a top priority. We strive to ensure that all reviews are fair, honest, and trustworthy. As we refine this experience, we welcome your constructive feedback within this thread.

Here are a few key points to keep in mind:

The updated review process will align with the guidelines outlined on our Review Process Page. These guidelines are in place to ensure that all reviews on Zazzle are fair, honest, and trustworthy, while ensuring compliance with relevant guidelines.
Reviews can only be removed for reasons specified on our Review Process Page.
We recognize that you may have feedback, and we want you to know that these updates are being rolled out in phases to allow for careful testing and adjustments.

We believe these improvements will benefit our entire community—Customers, Creators, and third-party Manufacturers—by promoting open communication and fostering continuous improvement.

Thank you for your continued support.

Creator Team

Reviews

whimsywhim — Fri, 07 Jun 2024 20:36:53 GMT

Zazzle has a stunningly unfair review process, and I strongly urge the powers that be to change it as soon as possible. In this, Zazzle is even worse than eBay which is notorious for treating sellers horribly. Even eBay, of all places, allows sellers to respond to reviews and deletes unjust reviews.

The customer, himself, can make changes to images and blame the seller for it. The customer can blame the seller for shipping when we don't ship the item. The customer can blame the seller for the production value when we don't produce the item. The customer can blame the seller for printing problems when we don't print the item. The list goes on.

Zazzle expects sellers just to take the unfair and harmful reviews and be okay with them. Meanwhile, potential buyers believe the seller is to blame and won't buy. This is clearly an untenable and egregiously unfair situation for the seller.

Zazzle, you need to proactively change the review process. At the very least, allow sellers to respond. Zazzle should also delete unfair reviews. Please do at least what eBay does.

Re: Reviews

PGCo — Fri, 13 Sep 2024 22:30:55 GMT

I agree with this 100%. Has anyone reached out to you regarding this matter?

I am dealing with customer support, the content team, and I just emailed the team.creator@zazzle.com email to see if anyone is actually willing to help. I am astounded by how absurdly unjust this is to the designer. I have 3 reviews which I am trying to get removed from my store/profile which are clearly about print quality - not the design. Everyone in customer support agrees with me that the reviews are not my fault or my responsibility, yet no one is able to remove them....? It makes no sense. I am hoping someone will get back to me on this....so disappointing.

Re: Reviews

CLC — Sat, 14 Sep 2024 02:40:28 GMT

The review system makes no sense. I received a 2 star review because the customer ordered milk chocolate covered oreos when they wanted white chocolate. I don't do cover photos as I think they are confusing and do not represent the products accurately, so the product picture was the standard marketplace sideways photo of a milk chocolate covered cookie, nothing but brown showing, no way the customer did not know it was milk chocolate, so the review itself was just ridiculous, but I was so upset that I deleted the product as it only had a few views and posted it again with a small change. The review stayed on my account for about 6 weeks then disappeared, which was awesome, but it should have never posted to my store as it was not about the artwork.

Re: Reviews

whimsywhim — Sat, 14 Sep 2024 13:48:01 GMT

It is inexplicable how anyone can try to justify keeping reviews on designers' stores that have to do with print quality, mailing issues, etc - all issues that have nothing to do with the design or designer. Zazzle needs to take a hard look at their review policy. Unjust reviews really harm the designer.

Re: Reviews

Missstacey — Wed, 18 Sep 2024 16:56:13 GMT

If that alone wasn't bad enough, it now also directly affects the collection ranking. Imagine having a collection with 30+ items which customers keep purchasing time after time without leaving any reviews, but then only one person isn't satisfied with the print quality/shipping/size etc. and they give you 1 star - it is now publicly displayed as an official collection ranking. Would you purchase an item from a collection that has a 1 star ranking? I would not. I also do not understand why our items/collections should take a hit when in 99% of cases the review has absolutely nothing to do with the design.

Re: Reviews

whimsywhim — Wed, 18 Sep 2024 17:53:03 GMT

Holy moly! I didn't know that about the collection. Good grief. There needs to be a massive revamping of the review process.

Re: Reviews AB test

Cat — Wed, 18 Sep 2024 22:41:16 GMT

OK... maybe you could clarify what has changed?

The primary concern I have with reviews is that a negative review as a result of a printing or shipping problem gets attached to the design - even though these are things over which the designer ("creator") has zero control. It seems like there should be some way to separate that out so that designs don't get "marked down" for issues that have nothing to do with the design.

Re: Reviews AB test

ColsCreations — Thu, 19 Sep 2024 02:48:35 GMT

OK... maybe you could clarify what has changed?

Been a long time since I did a review but I thought there were distinctly different sections on it for product, printing and a third thing I don't recall. But I just went to do a review now on a card I ordered quite a while ago and this is what the review screen looks like now:

You can give it a star rating and there's one and only one text field for entering your general comment. I typed my comment (positive) and hit the Submit button. There was no preview/review screen after, it goes directly to a 'thanks for submitting your review' screen.

( @Fiorenzo , it was your card I reviewed so keep on eye on your notifications to see how it shakes out on your end.)

Re: Reviews AB test

MOM — Thu, 19 Sep 2024 03:30:03 GMT

I'm glad I haven't gotten a written feedback yet (if I had they would have been both negative for an error not made by my self - fortunately I have received them in both cases verbally). As a seller I totally dislike the feedback system (I was an eBay seller for many years with many, many years at 100% positive) and as a customer I wonder how many feedbacks are actually really true and not just bought as I have been approached more than once with the offer to make money posting fake reviews on Amazon.

In my honest opinion reviews are overrated and do little to foster trust because they often depend on to many moving factors (mood of poster, unlucky things happening during production, transport or delivery and so fort).

Re: Reviews AB test

Cat — Thu, 19 Sep 2024 03:39:43 GMT

To be clear, it's not just that I'm whining about the "it's not fair" aspect of things, it's also that when customers read reviews that reference shipping or packaging issues, it furthers the misunderstanding that Zazzle is simply a storefront app like Etsy, and that we "creators" are actually "sellers" who handle the manufacturing & shipping. I just don't think that misunderstanding helps anyone - not designers, not customers, and ultimately, not Zazzle.

Re: Reviews AB test

Cat — Thu, 19 Sep 2024 03:41:40 GMT

Thanks for the info. Not sure if this is an improvement or not - I suppose it invites less feedback on issues like printing or shipping - I guess maybe that's good? Hmmmm...

Re: Reviews AB test

Cat — Thu, 19 Sep 2024 03:44:25 GMT

I agree that reviews can be very "iffy" especially when some retailers actively do things to try to get people to take down negative reviews etc. I'm not sure how much customers rely on them when choosing between designs - it does seem like the information provided is often not helpful in that regard.

Re: Reviews AB test

CreativeLeahG — Thu, 19 Sep 2024 08:59:24 GMT

Currently reviews aren’t visible on our profile or product page is that changing? We can see them but customers can’t.

Meanwhile I do agree customers should be able to accurately reflect the quality of the product and slow shipping speeds etc. It’s no different to any other review system and the design itself is visible before they order so I have no concerns re that.

The review isn’t a reflection of us or our work it’s a reflection of production and shipping and product quality. In the main..

Re: Reviews AB test

MOM — Thu, 19 Sep 2024 09:12:16 GMT

@CreativeLeahGThanks Leah. I honestly thought the reviews would attach to our profiles. Phew. So why did I read posts in this forum by creators being upset about negative feedback?

Re: Reviews AB test

Cat — Thu, 19 Sep 2024 12:44:45 GMT

I agree that customers deserve an honest review that reflects their whole experience, but I do think that there's an issue here because while the review might not be visible on our profile page, they're definitely visible on the individual product pages, and the average of all the reviews for products in a collection (in terms of number of stars) is how the stars for the collection are calculated.

Here's a quick example - just a random wedding invitation: Simple Black & White Overlay Photo Wedding Invitation | Zazzle if you scroll down below the "about" section, you can read all of the reviews for this product/design and there are ratings for things like product quality, print quality, and shipped on time. It makes it seem like those issues are reflective of that invitation in particular, and since even a popular product like this one has relatively few reviews, one negative one can really impact the score. Look at this review - seriously, this is NOT the designer's fault - I'm betting it was one of those situations where they messaged the designer and never even read the response.

I'm still unclear how this new system will impact what the customers see. It sounds like customers are no longer asked to comment on things like print quality and shipping - so maybe this is a positive change in that respect? Hard to say. And who knows how many customers actually see or read those reviews - clearly you didn't realize they were there, so maybe nobody else does either?

I dunno, maybe it's not worth worrying about, but it is a bit of a gut punch when you see that one of your collections has a star rating of 3 because one person got angry about a shipping problem or something like that.

Re: Reviews AB test

MOM — Thu, 19 Sep 2024 13:18:34 GMT

@CatThanks so much for this example. Yes, indeed, I think I never have scrolled all the way down past the description so I never have seen a review on an item. I just have seen some folks having left reviews for others and that I have found on their pages under Reviews.

So in your example that item had 30.3K people viewing this design but there were only three reviews in total - one per 10K of visitors. That's an eye opener for a former eBay seller. Wow!! in my experience about every tenth person (buyers) leaves feedback on eBay and I was a seller who never have asked my customers to leave feedback because I think they did their part already - they paid so why bother them to write a review. But I think for Etsy folks feedback is very likely a tad more important by design.

PS: Since the beginning I have found it kinda amusing when I saw the diamond on one of my own items:

This item is trending! 10 people viewed this design - it's encouraging for a new seller like me of course but holy smoke - 30K of lookers with the very same diamond. Wow!! I can only guess how often this sellers item has sold, smile.

Re: Reviews AB test

LMGildersleeve — Thu, 19 Sep 2024 13:55:02 GMT

@CreativeLeahG Leah, when I am logged out I see the new format on my desktop. I see my towels review on the product page. Or course, on our Members profile still shows our own reviews but not other customers.

This is the testing view.

Re: Reviews AB test

Cat — Thu, 19 Sep 2024 14:16:11 GMT

Full disclosure, I edited my post to use a different example because it better illustrated the point I was trying to make. Just clarifying in case anyone sees this and is confused because the numbers don't add up. Here's the product I originally linked to: Pine Tree Forest Rustic Watercolor Themed Wedding Invitation | Zazzle

Re: Reviews AB test

CreativeLeahG — Thu, 19 Sep 2024 20:34:28 GMT

As a consumer, this review tells me the customer made a mistake