Reviews AB test

Scott
Community Manager
Community Manager

Hi Zazzle Creators,

We want to inform you about an upcoming A/B test aimed at improving the product review process. These changes result from thorough research and are designed to enhance customer trust and overall experience on the Zazzle marketplace.

Our commitment to transparency and integrity remains a top priority. We strive to ensure that all reviews are fair, honest, and trustworthy. As we refine this experience, we welcome your constructive feedback within this thread.

Here are a few key points to keep in mind:

  • The updated review process will align with the guidelines outlined on our Review Process Page. These guidelines are in place to ensure that all reviews on Zazzle are fair, honest, and trustworthy, while ensuring compliance with relevant guidelines.
  • Reviews can only be removed for reasons specified on our Review Process Page.
  • We recognize that you may have feedback, and we want you to know that these updates are being rolled out in phases to allow for careful testing and adjustments.

We believe these improvements will benefit our entire community—Customers, Creators, and third-party Manufacturers—by promoting open communication and fostering continuous improvement.

Thank you for your continued support.

Creator Team

62 REPLIES 62

Cat
Honored Contributor III

OK... maybe you could clarify what has changed? 

The primary concern I have with reviews is that a negative review as a result of a printing or shipping problem gets attached to the design - even though these are things over which the designer ("creator") has zero control. It seems like there should be some way to separate that out so that designs don't get "marked down" for issues that have nothing to do with the design.

____________________
Cat @ ZingerBug Designs

ColsCreations
Honored Contributor II
OK... maybe you could clarify what has changed?

Been a long time since I did a review but I thought there were distinctly different sections on it for product, printing and a third thing I don't recall. But I just went to do a review now on a card I ordered quite a while ago and this is what the review screen looks like now:

newReviews.jpg

You can give it a star rating and there's one and only one text field for entering your general comment. I typed my comment (positive) and hit the Submit button. There was no preview/review screen after, it goes directly to a 'thanks for submitting your review' screen.

( @Fiorenzo , it was your card I reviewed so keep on eye on your notifications to see how it shakes out on your end.)

Store IconStore IconWebsite IconFacebook IconDiscord IconPinterest IconInstagram IconNight Cafe IconOut of Stock List

Cat
Honored Contributor III

Thanks for the info. Not sure if this is an improvement or not - I suppose it invites less feedback on issues like printing or shipping - I guess maybe that's good? Hmmmm...

____________________
Cat @ ZingerBug Designs

jophb
Contributor II

I think it does the opposite because they have no direction or parameters. And if they are mad at what they received or experienced, they are going to unload there.  If it separated the design and other parts of the experience, then I think we as creators would not get hit. I have some reviews that are strictly about Zazzle related issues since September.

Fiorenzo
Valued Contributor II

Just read your post and checked. Didn't expect the review to be already published since it always took ages. But it is there. Both on my review page as well as on the product page under Reviews/This design.

Your review only shows the "About the product" text and the "Recommended" and "Shipped on time" points. No "About the print" text anymore, and no "Product Quality", "Print Quality", and "Recommended For".

Edit: Your review is missing in the "Other reviews" section on the product page. With sort set on "newest first" and one third-party review newer than yours.

──────────
FX GRAPHICA Art & Design | PET’S DREAMLANDS » Store - Facebook | CONTACT: fio@fxgraphica.com

ColsCreations
Honored Contributor II

Since I can't find any trace on my end of having made this review, I had to search up the product page to see it.  There's a screenshot in previous post of what the input page looked like. What I entered under "Care to provide details?" shows on the posted review as "About the Product".  The posted review includes "Shipped on Time - Yes" and "Recommended - Yes" and while both are true, these were not options anywhere on the review page when submitting it.

Store IconStore IconWebsite IconFacebook IconDiscord IconPinterest IconInstagram IconNight Cafe IconOut of Stock List

Fiorenzo
Valued Contributor II

Since the changes are still in the making, this will be a (supposedly/hopefully) temporary effect of the usual let's put it live and see how it works approach we know from other things. I wouldn't waste time on this.

──────────
FX GRAPHICA Art & Design | PET’S DREAMLANDS » Store - Facebook | CONTACT: fio@fxgraphica.com

Cat
Honored Contributor III

To be clear, it's not just that I'm whining about the "it's not fair" aspect of things, it's also that when customers read reviews that reference shipping or packaging issues, it furthers the misunderstanding that Zazzle is simply a storefront app like Etsy, and that we "creators" are actually "sellers" who handle the manufacturing & shipping. I just don't think that misunderstanding helps anyone - not designers, not customers, and ultimately, not Zazzle.

____________________
Cat @ ZingerBug Designs

Totally agree with this!

MOM
Valued Contributor

I'm glad I haven't gotten a written feedback yet (if I had they would have been both negative for an error not made by my self - fortunately I have received them in both cases verbally). As a seller I totally dislike the feedback system (I was an eBay seller for many years with many, many years at 100% positive) and as a customer I wonder how many feedbacks are actually really true and not just bought as I have been approached more than once with the offer to make money posting fake reviews on Amazon.

In my honest opinion reviews are overrated and do little to foster trust because they often depend on to many moving factors (mood of poster, unlucky things happening during production, transport or delivery and so fort).

San FranciscoRSSSan FranciscoBenableBlueskyFacebookflickrInstagramLinkedinPinterestRedditTikTokThreadsvimeoXYouTube

Cat
Honored Contributor III

I agree that reviews can be very "iffy" especially when some retailers actively do things to try to get people to take down negative reviews etc. I'm not sure how much customers rely on them when choosing between designs - it does seem like the information provided is often not helpful in that regard. 

____________________
Cat @ ZingerBug Designs

CreativeLeahG
Honored Contributor III

Currently reviews aren’t visible on our profile or product page is that changing? We can see them but customers can’t.

Meanwhile I do agree customers should be able to accurately reflect the quality of the product and slow shipping speeds etc. It’s no different to any other review system and the design itself is visible before they order so I have no concerns re that.

The review isn’t a reflection of us or our work it’s a reflection of production and shipping and product quality. In the main.. 

MOM
Valued Contributor

@CreativeLeahGThanks Leah. I honestly thought the reviews would attach to our profiles. Phew. So why did I read posts in this forum by creators being upset about negative feedback?

San FranciscoRSSSan FranciscoBenableBlueskyFacebookflickrInstagramLinkedinPinterestRedditTikTokThreadsvimeoXYouTube

Cat
Honored Contributor III

I agree that customers deserve an honest review that reflects their whole experience, but I do think that there's an issue here because while the review might not be visible on our profile page, they're definitely visible on the individual product pages, and the average of all the reviews for products in a collection (in terms of number of stars) is how the stars for the collection are calculated.

Here's a quick example - just a random wedding invitation: Simple Black & White Overlay Photo Wedding Invitation | Zazzle if you scroll down below the "about" section, you can read all of the reviews for this product/design and there are ratings for things like product quality, print quality, and shipped on time. It makes it seem like those issues are reflective of that invitation in particular, and since even a popular product like this one has relatively few reviews, one negative one can really impact the score. Look at this review - seriously, this is NOT the designer's fault - I'm betting it was one of those situations where they messaged the designer and never even read the response.

Cat_0-1726749693828.png

I'm still unclear how this new system will impact what the customers see. It sounds like customers are no longer asked to comment on things like print quality and shipping - so maybe this is a positive change in that respect? Hard to say. And who knows how many customers actually see or read those reviews - clearly you didn't realize they were there, so maybe nobody else does either? 

I dunno, maybe it's not worth worrying about, but it is a bit of a gut punch when you see that one of your collections has a star rating of 3 because one person got angry about a shipping problem or something like that.

____________________
Cat @ ZingerBug Designs

MOM
Valued Contributor

@CatThanks so much for this example. Yes, indeed, I think I never have scrolled all the way down past the description so I never have seen a review on an item. I just have seen some folks having left reviews for others and that I have found on their pages under Reviews.

So in your example that item had 30.3K people viewing this design but there were only three reviews in total - one per 10K of visitors. That's an eye opener for a former eBay seller. Wow!! in my experience about every tenth person (buyers) leaves feedback on eBay and I was a seller who never have asked my customers to leave feedback because I think they did their part already - they paid so why bother them to write a review. But I think for Etsy folks feedback is very likely a tad more important by design.

PS: Since the beginning I have found it kinda amusing when I saw the diamond on one of my own items:

This item is trending! 10 people viewed this design - it's encouraging for a new seller like me of course but holy smoke - 30K of lookers with the very same diamond. Wow!! I can only guess how often this sellers item has sold, smile.
San FranciscoRSSSan FranciscoBenableBlueskyFacebookflickrInstagramLinkedinPinterestRedditTikTokThreadsvimeoXYouTube

Cat
Honored Contributor III

Full disclosure, I edited my post to use a different example because it better illustrated the point I was trying to make. Just clarifying in case anyone sees this and is confused because the numbers don't add up. Here's the product I originally linked to: Pine Tree Forest Rustic Watercolor Themed Wedding Invitation | Zazzle

____________________
Cat @ ZingerBug Designs

CreativeLeahG
Honored Contributor III

As a consumer, this review tells me the customer made a mistake

@CreativeLeahG  Leah, when I am logged out I see the new format on my desktop. I see my towels review on the product page. Or course, on our Members profile still shows our own reviews but not other customers.

This is the testing view.

towel review.png

This seems like a legitimate review? 

Out of 505 reviews I only have

one '1 star review' (customer writes it was their error that caused the issue!)

and

6 x 2 star ratings 

7 x 3 stars

Everything else is 4 and above, so % wise I'm not going to worry about this. Meanwhile some reviews do indicate changes are required (too small/dark text etc.) majority though user or production error.

 

Yes Leah, it is. She writes "I really want my money back". I confirmed with Zazzle that she was helped further. There is nothing I can do about this low review so I'm not worried. It's actually the very first review I've had that is below 4 stars.

So this is how the new test looks like. One box with comment. Zazzle combined the "About The Product" and "About The Print" into one box.

alissag
Contributor III

With customizable products the reviews can be misleading. I had someone leave a review on a yard sign. They complained that it was a different front on the front of the sign than on the back. So, naturally I checked to make sure I hadn't somehow goofed on that. Nope. Same font on both sides. My guess is that they customized their sign with a different font, but only did so for one side of the sign assuming it would transfer over to the reverse. So, as a designer I was able to figure out what happened, but the average customer seeing that would assume there was something wonky with the product and move on to another item.

James
Community Manager
Community Manager

Hi @alissag , to help avoid this on products that have duplicated sides or text; once you make a text box a template, you can assign the same URL parameter to each duplicated layer. For example, using "text1" in the box below for both front and back text boxes will update/change all connected layers. 

James_0-1727259919938.png

 

 

Jermolinaart
New Contributor III

Are there any updates on reviewing the review system, especially regarding collections? I’ve been receiving 1- and 2-star reviews because items were not delivered. Some customers are even accusing me of not sending the items, even though I’m only the designer. This has resulted in my bestselling collection getting unfairly low ratings, which doesn’t seem right.

whimsywhim
Valued Contributor

The review process still allows the customer to ding the seller regarding actions the seller is not responsible for, i.e. printing, shipping, what the customer herself did, etc.  If there is full transparency, then the seller should have the chance to respond.  Further, the review should be taken off completely when it is not the seller's fault.  

Reviews can hurt the seller.  It is exceptionally egregious when it is not the seller's fault.  

Someone on mine had a "white line down the side". I designed it to cover fully and beyond like always with full design prints (I've been here since 2012 and went through all the changes since). I could not figure out how to fix it because it must be something in QC or the printing process or just how the product edges are? 

If buyer altered it in any way, I cannot know what they have done 😞 that might have caused the white line on all over/full color.

So the whole review looks bad because of something I have no clue about.

SheaPrints
Contributor III

For one of my collections I've received eleven reviews in total:
7 x five star reviews
1 x four star review:
3 x one star reviews:

The 'one' star reviews only spoke about shipping problems/print quality. The 'four' star review was happy, but said it looked a bit dark. The design is black & white, so I think it must have been their photo.

My star rating for this collection is now 4.1 - is that based on all the reviews I've received?
This collection includes some of my top sellers, so eleven reviews represent a tiny fraction of the total sales!

Only some customers realize that the designer isn’t responsible for the quality of the prints or the shipping process.  I often receive messages inquiring if I can assure a delivery date or modify their shipping address.

I think reviews are helpful in many ways, it's always interesting to see how customers adapt a design. Overall, they've been really good. But, it's frustrating if our 'Collection' Star Rating is affected by low ratings for 'Print Quality' and 'Shipping'. I wish there was a separate rating for this. If it was separate, it wouldn't damage the design or collection rating.

I totally agree with you! Reviews are definitely helpful for getting a sense of what customers think, but it’s so frustrating when issues like print quality or shipping affect the overall collection or design rating. Like you said, it's not really a reflection of the design itself but more about external factors. It's especially tough when one customer leaves multiple low ratings due to something out of your control, like shipping delays, and it drags down the entire collection’s rating.

The current system seems to unfairly punish collections, especially for products that are well-loved and have great designs.

I hope they consider updating the system to make it more fair for creators!

LauraLee
Contributor III

Honestly, I don't believe these improvements will benefit like Zazzle thinks.  I believe the benefit comes from excluding Zazzle as the primary responsible one for the quality of the products themselves and the shipping it does.  Therefore, not having to address the issues we are tolerating with negative reviews that are based off of these.  And technically, those are the ones that are hurting designers [creators].  As far as third-party manufacturers, they're protected behind Zazzle's quality assurance standards.  
I, for one, have been educated before I buy anything online to review the negative comments first.  Why??  To point to who and what is the reason for my purchase online.  Do I trust this product and this business who is selling it?  
So, to agree with this review process is nothing more than agreeing to shoot myself in the foot.  You first did it with our "chat" message system by having customers think we are the producer of these products, and maintain that an auto text message will reflect that "we're just the designer [creator] and to contact customer service" not mentioning Zazzle by name.  And this follows up with not taking any responsibility for the product themselves or the shipping?  The two are clearly cya, imho.  But, I'm going to take the stand and say, that I think this is wrong.  You ARE hurting the designers here.  You are putting the blame on us.  Take a look at this review.  Are you saying that this review is for the 3rd party?  If so, why do I have it on MY reviews if this has nothing to do with my design?  If this review is for the 3rd party, yet you can hide the 3rd party manufacturer pretty well from the customer, who are you thinking this affects?  If it were me, and it's not, might I suggest that the manufacturer is not hid from customers.  So that they can at least make this a FAIR playing field for us as designers and them as 3rd party manufacturers.  It's ultimately Zazzle's responsibility as well as hurting Zazzle's reputation as well.  And that's why I have to stand up.  I like Zazzle, and I don't want to see anyone hurt by doing this, but to me it's definitely going to lead to hurting creators instead of the intended.  The 3rd party name is no where even on this.  'Could at least put the name of the 3rd party to ones like this.    

Screenshot 2025-01-24 at 11.19.29 AM.png

 

For anyone interested....check out this page. https://www.zazzle.com/store/avery_products_corp/reviews?sort=RatingAsc 

Maybe designers can think about these one stars if they were more transparent to us.

Just read all of those terrible reviews.  I too always look at the negative reviews, in addition to the good ones and any pictures that may be posted on sites by people, to get an overall feel of the item before ordering on any site.  But it's not good reading through alot of those reviews regarding the binders from that company being sold here, or that they're pointing out the artists regarding quality, print, shipping, etc. as if it's their mistake and they caused it all, and by saying they'll never order from them again, etc.  Sounds like they think this is like an Etsy or Ebay type place where the artist designer...does it all.  I never ordered binders here, but over the years ordered different types of items from Zazzle and most times the print and everything was good, shipping too.  If not, customer service was helpful to resolve an issue. 

But there are indeed times that a customer may also make their own mistake with any type of item whether with personalization or misspelling error, own poor image quality, etc., and I've seen reviews in the marketplace where customers have acknowledged that, and that customer service helped them out and their orders were then received satisfactory or excellent.

But this particular type binder item issue (and any other product with similar problems) should be looked into more, I agree.  And it needs to be made much clearer to customers that the artists designers (creators) who do have their own stores within the Marketplace here, do not actually print, manufacturer or ship, etc. any items from Zazzle or its third party vendors.  As it is currently, some customers it seems feel like placing the blame on the artists here is the right thing to do for their overall dissatisfaction from printing to shipping of any items, and also think they're like customer service employees, which of course they're not.  Hope it can all be resolved, especially since I think Zazzle itself is still a great company with so much to offer in the marketplace with their vast array of items to design on, and to personalize.

Barbara
Esteemed Contributor

After looking at both the negative reviews AND the positive reviews, I came away with the feeling I was seeing the same thing as on Amazon, which is that customers don't always realize what they're ordering. We, as designers, should never mimic a texture and then highlight the appearance of that texture in the title. I now appreciate those little flags Zazzle places on our products such as there being no real lace or metal or whatever. Either the customer doesn't notice this or doesn't believe it.

Then there are those customers who don't understand that the delivery service and Zazzle aren't the least connected. At least on Amazon, the customers know how awful delivery can be.

But how is it that two customers can buy the same product and have opposite reactions to it? Expectations?

Most important to us is that customers often don't know we are not the creators of the product. The change from "designer" to "creator" was one of Zazzle's worst decisions. Even if this hadn't been done, we probably shouldn't be given as much emphasis because it leads to customers believing they're buying from us. Really, do we need our names up in lights? Why should we be blamed for the quality of the product? Do we blame Amazon for the quality of a skillet? Nope. But here, we can be blamed for the quality of a binder.

Colorwash's Home

Cat
Honored Contributor III

"Most important to us is that customers often don't know we are not the creators of the product."  This is the key issue - and it impacts so many things, beyond just reviews, it's Zazzle's Achilles Heel as far as I'm concerned. 

I just spent half an hour chatting with a panicked bride-to-be who assumed as MANY (if not most) customers do, that she was filling out a form to tell me how she wanted me to customize her invitations, rather than understanding that it's a WYSIWYG (what you see is what you get) system and that she herself was doing the customization. Fortunately, this one contacted me a few hours after placing the order rather than waiting until they arrived printed incorrectly, so there was time to cancel and redo it without too much delay, but still. It just causes SO MUCH UNNECESSARY UPSET.  

 

____________________
Cat @ ZingerBug Designs

LauraLee
Contributor III

@Cat,  I think you nailed it.  Many times on Amazon & WalMart, if I'm looking for something I've never bought before...I go straight to the reviews.  First the negative, but also the 5 star ones too.  To indicate how the product holds up and how the shipping is handled....why?  Because I'm paying for it.  Amazon & WalMart at least understands the impact it has with many 3rd party sellers, how it can hurt their own reputations in the indirect way.  So, it gives the customer awareness that 'this store or that store' is selling off product quality and in some cases handling the shipping ( I've seen cases where it obviously comes from a foreign country - and you can expect long delays).  But, you've got to be knowledgeable and research things [even those lowest prices or get burned]. 
And really that's why I had to speak up.  If Zazzle is not going to help our business as creators here, then we're looking at a short term for Zazzle overall.  Because I've seen it in the ratings, it's turning customers away.  Customers will be customers, I agree, But were they given the chance to research products properly?  Openly, with the product manufacturer's name and link to their reviews?  After thinking about just one product, have I done my own research here on other products?  To know what products to design on and keep away from??  Perhaps not very much, as I should have.  

Totally agree. This only hurts us 😞 

jophb
Contributor II

There REALLY needs to be a way for Zazzle to separate the review reasons.  IE what we as designers control, and what Zazzle controls.  It is incredibly unfair that our designs or shops get 1 star reviews because an item did not arrive on time or because an envelope doesn't fit the invitation properly.  And on top of that, the reviews can then harm an entire collection because it only happens to have one 1 star review in it.

And perhaps there needs to be a more obvious note in the reviews that the design is by an independent creator/individual human, and not just a product of Zazzle.

whimsywhim
Valued Contributor

Zazzle reviews need a way for us designers to respond.  They also need to be removed when the buyer blames us for mailing / printing problems, etc that are totally out of our control.

totally! Or at least not assign us the star rating. Or at the VERY least don't let it screw over an entire collection.

whimsywhim
Valued Contributor

Zazzle has a stunningly unfair review process, and I strongly urge the powers that be to change it as soon as possible.  In this, Zazzle is even worse than eBay which is notorious for treating sellers horribly.  Even eBay, of all places, allows sellers to respond to reviews and deletes unjust reviews.  

The customer, himself, can make changes to images and blame the seller for it.  The customer can blame the seller for shipping when we don't ship the item.  The customer can blame the seller for the production value when we don't produce the item.  The customer can blame the seller for printing problems when we don't print the item.  The list goes on.

Zazzle expects sellers just to take the unfair and harmful reviews and be okay with them.  Meanwhile, potential buyers believe the seller is to blame and won't buy.  This is clearly an untenable and egregiously unfair situation for the seller.  

Zazzle, you need to proactively change the review process.  At the very least, allow sellers to respond.  Zazzle should also delete unfair reviews.  Please do at least what eBay does.