An interesting roundtable about criticism in podcasting, particularly in podcast fiction. This is further wellspring from the #podcrit Twitter discussion.
I encourage you to go listen to this discussion; I’ve got a few comments and quite a bit more thinking below…
Let Me Tell You That You Screwed Up
There is definitely an interest in having more critics, but a suggestion that they need to not be podcasters — that they be outside of the politics of the community.
On standards, there is the question of where the standards come from — who made them? Who forces them to be? Who enforces them? The fact that podcasting has no gates, has no barriers, presents a problem.. Does the free market enforce these standards? But how does the free market work when everything is free?
There is definitely some sentiment that we have to “get our act together” to “earn some legitimacy” so that we will be taken seriously by paying markets.
Are we really treating each other with kid gloves? Where does that come from? Are we worried about discouraging amateurs? We’re (almost) all amateurs here, but some of us are more experienced than others. There definitely seems to be a need to be tough on veterans.
Levels of Criticism
Certainly, there is one kind of criticism that we all can accept: pure blunders, things that are obviously wrong, like repeated lines, tracks screwed up, etc.
The next level is more of craft: is the audio clear? Does the recording sound good? Is the editting clean, or is it sloppy?
The next one is trickier: criticism of content. This seems to be where most people get a little more leery. Again, when there are obvious mistakes or mis-spoken words, that’s easy to take — and should be pointed out enthusiastically. I’ve gotten called out on not doing enough research when I do a piece for my other show (TWS) — and it was very reasonable, and made me really think about how I produce the show — and how seriously people were actually taking my words.
But What About Podcast That Aren’t Books?
The discussion on the roundtable was specifically for podcast novelists, and they have a distinct advantage over most other types of podcasts: there are generally more revision steps involved in writing fiction and producing audio versions of the fiction. Those revision steps are external as well as internal, involving beta-readers and beta-listeners who serve as vetting services for podcast fiction.
In most other podcasts, however, there seems to be between 1 and 5 stages:
- Do research.
- Write the segments.
- Record the segments.
- Edit the show together.
- Review the edited show, maybe returning up one or more stages.
Most podcasts have stage 3 and a little bit of stage 4. I think that very few have stage 2, even if they have stage 1. Most shows, even if they have research, prefer to record “off-the-cuff” for a more natural expression. Some shows have a stage 5, but I think that they don’t do that very often, due to time.
For some examples from my own work:
- For TWS, I go through stages 1, 3 and 4. I don’t have time to do 2 or 5, and trust that my editing style and skills put the show together reasonably well, so even stage 4 is very abbreviated, more on the level of “assembling” than “editing”. I don’t review my show for content at all — a step that I sometimes regret, but one I simply cannot afford to do.
- For WOL, I go through stages 3 and 4, and occasionally 5. I don’t do any research at all, although I might get some ideas at hand. I have experimented with stage 5 during the editing process, in which I react to my own original recording with clarifications, corrections or extensions. Again, my stage 4 editing is more like assembling, bolting a front-end and back-end on and leveling things.
- For UP!, I started by doing all 5 stages. The first episode was done that way, but there was a problem: I could not write them fast enough (2). I found it tiresome and passion-killing. Now, I tend to also skip step 5.
- For the work I did for The 9th Heroescast, it generally followed all 5 stages for any segments I contributed to the show; there were some “live” segments which generally followed the 1, 3, 5 pattern.
The most professional shows tend to follow all five steps, but they also tend to sound very staged. There seems to be a trade-off between the fluidity of live conversation and the perfection of scripting. A few shows that I have heard use heavy editing to craft the episodes as if they were written — The Dusty Show on WFMU is a fine example of some rather extensive editing used to craft a show, although the editing itself is really a part of the show.
I think most podcast producers let step 5 be done by their listeners, but rarely return to the podcast and do any updates — instead, they will put an update and/or comment in the next episode. The old one is, strangely, fixed in time — despite that it can be consumed at any future point as well. In part, this is understandable: it’s already been downloaded by the audience, so it doesn’t really live in any controllable space.
But What About Criticism For Non-Novel Podcasts?
I think we can and we must start thinking about standards for all podcasts. I would challenge the idea that standards imply a sort of authoritarian, “down-from-the-mount” dictatorship over the form. Rather, I think standards can be simply a recognition of good practices to strive for and bad practices to avoid.
I have not yet read “Expert Podcasting Practices For Dummies” by Tee Morris, Evo Terra and Ryan Williams, but even in skimming it, I can see that a number of these issues are at least addressed, usually in terms of “good” and “bad”. If we needed a start on standards, it might be argued that this is it. I’ll be digging into it over the next couple of days to reflect on it’s advice and what we might apply going forward.
(Aside: I admit that the reason I only recently got the book was a combination of arrogance (“I’ve been podcasting for 5 years, and doing (amateur) radio for 12! I know what I’m doing!”) and annoyance (“Any book for ‘Dummies’ is aimed at a beginner audience, despite the fact that it says ‘Expert’ right on it!”). I’m trying to correct that now…)
But where does the criticism for podcasts come from? It has been suggested that it must come from somewhere “else”, someone not entirely in the community — not a podcaster, someone without a personal stake it in, so that they aren’t mired in the politics. Or, perhaps, they should be anonymous, free to say what they like without backlash.
The path of a critic isn’t without peril: if you say something bad, you are bound to piss someone off. I suspect that most critics need a very thick skin, or anonymity — but, a critic who puts themselves out there is going to be listened to more than an anonymous poster, I believe.
This reminds me somewhat of the Wikipedia problem: to write an article for Wikipedia, you must have sufficient credentials to claim some familiarity with what you are writing, but you must not be directly involved in what you are writing, and can only cite sources other than your own expertise. The goal here, of course, is to have some distance to be able to write objectively about the topic. The real effect, of course, is to eliminate much of the expertise in the field, because in most cases to truly know something is to be involved with it, to have a stake in it..
And In The End, We’re All Individuals
I don’t have any conclusions for this post — it was originally going to be just a few lines of reaction to the roundtable! — but I feel like this will be one of those things to come up from time to time. What’s your opinion?
- Is this really a genre thing? Is criticism much more important in the genre of podcast fiction (and the sub-genre of podcast novel fiction particularly) than in other genres of podcast?
- Should we draft criteria for rating episodes, perhaps within the two broad categories of craft (how well the show is put together) and content (the material that the show has)?
- Do we put up with crap too much? Do we forgive too readily? Do we all need to be a little more direct with each other, and as podcasters, grow a little thicker skin?
- Am I missing the point here? Is this really about stuff we need to teach people to do right, or about calling out and criticizing the things that people have already done?
- Are there other steps in the production process to consider? Are there particular examples you would hold up as “good” and others you would hold up as “bad”? (Feel free to use my shows as examples of bad, if you don’t want to name other names..)
- Is this all just wishy-washy whistling in the wind? Is this a distraction from the real issues of quality control?
- Do we need to create some sort of artificial barriers to podcast entry? Do we need to erect ivory towers and let only the best podcasts enter (ideally completely devoid of any politics)? For an alternative description, should we create a “gold certification” given out by a qualified and respected body of judges (drawn from peers), to indicate those podcasts which demonstrate compliance to standards or exceptional production value/content value?
« Can I get a #podcrit over here? Part 2: the tech factor UP004: Podcast Idea: “24 Hours ‘Round The World” »
I’ve come to the conclusion that y’all are out of your cotton-picking minds.
Mark, you state regularly that “podcasting is not a genre”, and I agree. (though not wholeheartedly. Podcasting has developed it’s own radio-like format, but that’s a different argument.) Lets take that even farther and admit that podcasting can’t exist in a vacuum.
One thing nobody has mentioned in this whole #podcrit dustup that I’ve seen yet is that, to be a podcaster, you need to do 3, 4 or maybe even 5 things at once.
1. You need the technical skills to produce your content, be it audio, video or print, and create that final deliverable.
2. For video and audio podcasts, which are the vast majority, you need to be a performer or a director of performers, and sometimes both.
3. You need to be competent in something. A writer, a gamer, a musician, a mountain climber, a something. Your podcast needs to be about something.
4. You need to be a competent marketeer, networker or otherwise effective self-promoter, so you can generate a satisfactory audience for yourself.
5. You need to be enough of a net geek to figure out the podcast distribution mechanism, which can be simple or hard, cheap or expensive depending on what choices you make.
when someone decides to podcast, they can’t “just podcast”, they need to engage a whole lot of processes at once. We’re not entirely honest about this to the newbies, and I suspect that it’s because of all this hidden work that’s never explicitly described that a lot of new podcasters wash out before they hit double digets.
When a podcast gets critiqued, what part of it are you critisizing? All those parts require some sort of mentorship from the “elders”, and a certain degree of skill in the podcaster.
Excellent points!
So, when we say there’s a “low barrier to entry” for podcasting, it’s true: there’s a “low barrier to entry” — to produce terrible shows?
You do suggest that when I created my list of 5 stages of production, I left a couple out:
6. Show notes are produced.
7. Each episode is promoted.
Now, that list is meant to be per-episode, and likely the criticizable sections, rather than the full process (find podcast host, find blog host, set up blog, set up RSS feed, upload episodes, manage comments, etc). So, it’s not meant to encompass all the things that a podcaster does, but rather the parts that have the most visible effects for the outside world.
No one really cares to see the research I do for a show — or really even is impressed at how many hours of research I do — but rather, they see the end product, and the research is demonstrated. That’s unfair, from a producer’s perspective, but entirely fair from a consumer — and a critic’s — point of view; isn’t it?
I wonder if that’s where the softening of critiques comes in, especially from other podcasters: we *know* how much work is involved, and we automatically compensate our criticisms to encompass that hidden work..
I think we might want to separate “outside” versus “behind-the-scenes” criticism, the view from the consumer and the view of the real work, the tip from the body of the iceberg.
Thanks for the comments! Very thought provoking..
…
Oh, and the “genre” thing is another good discussion to have. I only mention it here, and kinda biffed the use of “genre” to refer to fiction podcasts versus non-fiction podcasts..
Really, I suspect that those will come to be “formats” or “kinds”, whereas “genres” tends to refer to the content, such as “science fiction”, “news” or “rock and roll”.
I’m not entirely sure if there is a new format being developed with podcast shows, or if is merely the relaxation of radio-like and TV-like standards. That’s worthy of it’s own post or sixty..
1)Is this really a genre thing? Is criticism much more important in the genre of podcast fiction (and the sub-genre of podcast novel fiction particularly) than in other genres of podcast?
“Fiction” isn’t a genre by my definition, but I think that when it comes to novels, I as a listener would like to have a reliable indicator of quality that is cleanly separated from a popularity contest.
2)Should we draft criteria for rating episodes, perhaps within the two broad categories of craft (how well the show is put together) and content (the material that the show has)?
Any single objective reveiwer can set their own criteria. I only ask that they be consistent. I’m not sure how anyone can make the rules for other people to judge by.
3)Do we put up with crap too much? Do we forgive too readily? Do we all need to be a little more direct with each other, and as podcasters, grow a little thicker skin?
I agree with the point that was made about it not being the job of content creators to publicly judge other content creators. There is no way to be objective. Different people will have different strengths. I WILL say that I think the rating system on a place like podiobooks.com is completely useless. Number of beginning-to-end listeners is a much better indicator of quality. Why is this? Because the ratings (as far as I can tell) are not anonymous. As a content creator, I will 5/5 star something I love, but I will never rate and comment on anything below that level. It is not out of fear of a criticism in return. It is because we are too small of a production community to risk hurting feelings, despite good intentions.
4)Am I missing the point here? Is this really about stuff we need to teach people to do right, or about calling out and criticizing the things that people have already done?
In a perfect world, I would be able to call out people what I would like to see improve without risking bad blood. If I ever critiqued something, it is because I would want to listen to the improved level of production from that person.
5)Are there other steps in the production process to consider? Are there particular examples you would hold up as “good” and others you would hold up as “bad”? (Feel free to use my shows as examples of bad, if you don’t want to name other names..)
My biggest pet peeve in novels is garbled narration when the narrator was also the producer. I won’t point fingers, but such things have caused me to stop listening on the spot. It shows a lack of effort, and my time is valuable.
6) Is this all just wishy-washy whistling in the wind? Is this a distraction from the real issues of quality control?
I think any discussion is good, even if it only makes a few producers think about how they can improve their quality. I will never accept “but it’s free” as a reason not to do the best you can. Think about it. the cost is the listener’s time. With all the podcasts out there, I’d rather listen to more things that have good quality and release slower than fewer shows that rush out content without taking the time to edit the writing and/or the performance. I would LOVE a better filter so i could find the good stuff.
7) Do we need to create some sort of artificial barriers to podcast entry? Do we need to erect ivory towers and let only the best podcasts enter (ideally completely devoid of any politics)? For an alternative description, should we create a “gold certification” given out by a qualified and respected body of judges (drawn from peers), to indicate those podcasts which demonstrate compliance to standards or exceptional production value/content value?
No. This would be a terrible idea, would reinforce cliques over quality, and would be a massive waste of time. The best I could suggest would be a group of invited producers (the full list of which would be public) who would be able to critique submitted podcasts anonymously (as in you don’t know which members wrote which review) to generate an honest quality ranking from people who understand the medium (an open system could be overrun by internet trolls). I have no idea how to organize something like this, but it is the only way that shows that need one last nudge to be better (even great) would ever hear constructive criticism from me.
– Is this really a genre thing? Is criticism much more important in the genre of podcast fiction (and the sub-genre of podcast novel fiction particularly) than in other genres of podcast?
I think it might be more important for podcast fiction, because these podcasts are more timeless. Someone starting a podiobook will go back and listen to _all_ of the episodes, whereas someone listen to a non-fiction podcast won’t go back more than a handful of episodes, or maybe listen to a few that attracts them by their title. The reason for that is also, imho, the number of episodes in non-fiction podcasts – they are likely to go on longer.
For me, the attention I give to fiction podcasts is much higher than non-fiction, just because I don’t want to miss an important plot point. This also makes me more aware of things I like or don’t like in the execution of the podcast.
To sum it up – I think mistakes in podcasting are much easier to forgive and forget in a non-fiction podcast.
– Should we draft criteria for rating episodes, perhaps within the two broad categories of craft (how well the show is put together) and content (the material that the show has)?
The distinction makes sense to me, but like I mentioned before, it makes a huge difference for me if it’s a podiobook or an entertainment podcast or an information podcast, etc. In short, what kind of podcast genre it is defines the importance of said factors.
– Do we put up with crap too much? Do we forgive too readily? Do we all need to be a little more direct with each other, and as podcasters, grow a little thicker skin?
I, personally, I don’t forgive too easily. And I don’t know many people who do. We just don’t talk about it openly. For example, I have given up on a podcast after 5min of listening to it, cause I couldn’t bear the voice of the host. Would I go out and say ‘that guy shouldn’t be put behind a mic’? No.
This is a very subjective observation – and even though many people think the same thing, this is not something you tell someone else.
As far as more objective criticism is concerned – I think once you podcast yourself, you forgive mistakes. Especially the ones you made yourself at one point. There are only a few people one holds to higher standards. But that’s politics again.
– Am I missing the point here? Is this really about stuff we need to teach people to do right, or about calling out and criticizing the things that people have already done?
This is an age old philosophical question – teaching or criticizing?
I know when I first started podcasting, i sought advice from people who knew what they are doing. And yes, my copy of “Podcasting for Dummies” (First Edt) looks really rugged…
Did I always listen to the advice? No. Did I enjoy making my mistakes? Sometimes.
Would I rather start listening to a podcast from a point on where the podcaster knows what they are doing? Probably, yes.
– Are there other steps in the production process to consider? Are there particular examples you would hold up as “good” and others you would hold up as “bad”? (Feel free to use my shows as examples of bad, if you don’t want to name other names..)
Of course, we are all impressed with a crystal clear sound, perfect levels and all the audio things one can imagine – from a perfect cast/cohost/guest, to sounds effects, music, etc. Somehow, I want to say – the good thing about a podcast is, if it fits.
If it’s a bunch of friends sitting together talking about their favourite TV show, I wouldn’t mind background noises as much as in a company podcast that tries to teach the listener something.
I will mention this again – because I think most people keep very quiet about it. There are people whose voices get praised, and rightfully so – but does ever someone dare to tell anyone else that they can’t listen to their voice? I admit it – I might be snobby on the subject. To go that step further – what would someone think if told such a thing? Is it objective enough to say “well, that’s your opinion”? Or does something like that become objective, if enough (or all?) people think the same thing?
Sorry Mark, I seem to ask more questions here than I answer 😀
– Is this all just wishy-washy whistling in the wind? Is this a distraction from the real issues of quality control?
This sounds to me like a contradiction – I think it is about the issues of quality control and not so much a distraction since it is a means to talk about the WHOLE question.
– Do we need to create some sort of artificial barriers to podcast entry?
No! Never. Njet. Nein. Non.
– Do we need to erect ivory towers and let only the best podcasts enter (ideally completely devoid of any politics)? For an alternative description, should we create a “gold certification” given out by a qualified and respected body of judges (drawn from peers), to indicate those podcasts which demonstrate compliance to standards or exceptional production value/content value?
There will always be a natural way of having a select few that the random many aspire to be. This is, for me, the situation at hand. But no, I wouldn’t force an elite out in the open and on a pedestal, like the Academie Française watching over the purity of the French language, dictating what is right and true. I would much rather see a natural outlet – those who are ‘up there’ know it, or get it told. For me, it would be more interesting for them to kind of ‘mentor” new podcasts and podcasters, or just come as guests on other shows or lend their voices. In the course of that, maybe throw in the odd advice or two.
The last question – isn’t that what podcast awards are for?
Maybe we should make a new one? Do we need more or should the ones existing be promoted better?
Sometimes, I think, having one of the ‘big players’ help and contribute, shows much more. But there’s politics in that as well, as usual.
Thanks for the comments! Questions are very fine – it’s not like we have the answers yet, nor do we even necessarily know the right questions yet..
So, it seems as though the notion of getting critiques of a non-fiction podcast is a bit of a problem. In particular, because the market is so large, consumers have extraordinary choice, so are more likely to just walk away rather than provide any negative criticism.
That seems fair, but how can a podcaster use the non-response of listeners to make themselves better?
Perhaps we do need somewhere we can submit our podcast for criticism.. Podholes tried this, but a single show providing criticism just won’t work, given how many podcasts there are out there; any single source of criticism or comment would quickly be overwhelmed, just as it quickly became apparent that the interview series of podcast411 was too narrow to continue (in my opinion, it’s death was inevitable).
But what about something addressing the common problems? We talk a lot about good practices — what about bad practices?
The latter half of my “ivory tower” idea was some idea of certification or a “stamp of approval” from a body that should really know better. I’m not a huge fan of awards in the “one winner” sense in podcasting, however, simply because it is too big a field in general. It would be like having a “best dog in the world” contest, where every dog in the world can compete, from professional show-dogs to the mutt down the street. Totally unworkable.
Rather, I can envision the notion of a standards level or rating system, administrated by a collection of (semi-anonymous) judges, providing badges that a person can put on their site indicating “platinum rated podcast”, “gold standard podcast”, “silver quality”, etc.
Of course, one of the things that happens there is that no one will want to display a less-than-stellar rating on their site.. My only response to that is to make it conspicuous by its absence, make not having a rating imply that you are beneath the worst scores, and to make a central location where it can be looked up anyway and easily (say, via a bookmarklet), like an IMDB for podcasts..
[…] here: Criticism in Podcast Fiction roundtable ← Is it me or is podcasting on fire? – AST […]