Update on TPM Community Tech
There is understandably continued frustration about some of the technical problems we're having w/ the community tools. I've been holding off on reporting back to you all because we've spent much of the last few weeks trying to get a handle on how and how quickly we can fix things. Believe me when I tell you we share your frustrations and then some. It is obviously not in our interest to have a site that's frustrating you, and we're doing everything possible to fix it. It hasn't been as quick as we'd like, but we're doing absolutely everything we can.
Let me give you a quick update on the details.
There are really four core tech bugs that have been tough for us to kill.
First, there's the problem of folks being logged out mid-session, causing them to lose posts they're working on or longer comments. Without getting too deep into the tech weeds, this is a problem in which there are two cookies being sent that are conflicting. One of them thinks you're logged in, so it gives you a window to compose posts and a "Hi Username!" greeting in the task bar, but the other doesn't so when you try to push content into the database it denies you access and demands that you sign in. The software designers told us this week that they figured out why this is happening, and they gave us a two week timeline for testing and implementing the fix.
The second tech bug has to do with the lag time in between when a comment or post goes up and when it shows up on the site. This is, at its core, a server problem. We are, to be frank, happily overwhelmed with the number of people who are using the community tools across all the sites and the servers are having to rebuild pages at a rate that is pretty unprecedented for us. We've been working on getting up two new servers and should have them up, again, in about two weeks.
The third is a problem in which users are trying to request new passwords when they've forgotten theirs and the new passwords they generate don't work. We still don't quite understand why this is happening, but the software designers and the developers we personally work with are looking over the activity archives and email explanations from users and trying to figure it out. Until then, if this happens to you, email me at andrew at talkingpointsmemo dot com.
Speaking of archives, the last problem is that the reader blog archives from the old Cafe have not been migrated into the new system. Initially, we thought the database had just been corrupted and could be fixed fairly simply, but quickly we figured out that they just had not been migrated onto the new servers at all. And, just our luck these days, when we went back to our old server partners to go get the archives they told us that they'd literally just put them on a truck to be moved across the country (seriously, you can't make this stuff up). They told us today that it has finally arrived and will be up to start the transfer next Tuesday. So hopefully that will be completed by the end of the week next week.
Now, beyond these core problems we also understand that there are folks who were regulars at the old TPMCafe that miss some of the tools we had there. We have two priorities on this front. First we're planning to create a new page that will go on your task bar (next to "my blog" and "my profile") that says "my conversations." This page will be your own personal thread tracker and will aggregate all the responses to either your blog posts or your comments. That way, you'll always be able to see when someone has responded to you on any of the sites. Second, we're going to make sure folks can re-edit their own blog posts once they've been put up. That way, if something's gone wrong w/ the coding, or they've mis-spelled a word, they can jump back in and fix it right up. I don't want to over-promise on how quickly we'll be able to get either of these things together. We simply can't afford to have a design team work around the clock on development. But they are our priorities as soon as we've sorted out the more core tech bugs.
Let me say again that I understand and am sorry for the continued frustration some of you have felt. And I appreciate the fact that, at its core, it's born of the fact that you like what we're doing and want to be a part of it without having the slog through problems. We appreciate that deeply and want to fix things for you. Through all of it, we appreciate your continued patience and hope you'll keep working with us to get it all right. I'll update you when I know more.
Let me know if you have any thoughts or other bugs to report in the comments.
Andrew,
do you ever get the feeling you're a reincarnation of Sisyphus?
March 19, 2008 6:13 PM | Reply | Permalink
"Second, we're going to make sure folks can re-edit their own blog posts once they've been put up. That way, if something's gone wrong w/ the coding, or they've mis-spelled a word, they can jump back in and fix it right up."
Bad idea.
This is a political site. People argue. Those are both good things.
The ability to edit every post is essentially the ability to win every argument - or, at the least, not to appear a fool when it is only fair that one does.
I'd re-think that idea, if I were you. Edit Diaries? Yes.
Edit posts? No.
March 19, 2008 6:33 PM | Reply | Permalink
The way it used to work was that you could edit your comment, up to the point where someone else commented to your comment. At that point, it was set in concrete.
As I hear what Andrew wrote, it's not the comments that can be edited in future, only one's blog post, which makes sense to me.
March 19, 2008 7:00 PM | Reply | Permalink
We always used to be able to edit comment posts, for a limited amount of time. It didn't cause problems.
March 19, 2008 7:00 PM | Reply | Permalink
The way it used to be, we couldn't edit comments that had already been responded to. I imangine the newer edit feature would work in a similar way.
March 19, 2008 7:08 PM | Reply | Permalink
I think the editing is a good idea, it worked well before you remodeled the site. I can't remember anyone pulling what Jaime suggests.
What's a Diary?
March 19, 2008 7:01 PM | Reply | Permalink
Hmmm. Trouble in definition city...
The way I understand it, a thread is everything. In the case of this thread, from Mr. Golis' title:
Update on TPM Community Tech
...all the way to the bottom.
A Diary is the expository chunk at the beginning, where the Diarist (Mr. Golis again) begins the thread with the original thought(s) upon which members comment.
This, that I am writing now, is a comment - and I have always understood comment and post to be interchangeable terms.
I see now that on this site, the word 'Post' is used along the right side of the page as a label for what most of the bigger sites on the 'net (DailyKos, for example) would call 'The Recommended Diary List.'
The thing is, I guess, that Webster hasn't gotten his hands on it yet - and there really isn't agreement on what to call the individual components of a blog. Actually, there isn't much agreement on exactly what a blog is.
Andrew Sullivan calls his editorial snippets a blog - but I disagree. I don't think it's really a blog unless one can post comments.
Well, anyway, I still don't think comments should be editable. Takes the fun and the care out of it, for my tastes. But whatever. T'ain't my blog.
March 19, 2008 7:30 PM | Reply | Permalink
Jaime,
thanks for the info.
March 20, 2008 12:45 PM | Reply | Permalink
Andrew,
We could also use a way of flagging trolls and maybe a way to see who is recommending what posts... the old ratings system was transparent, at least.
March 19, 2008 7:03 PM | Reply | Permalink
Hey Andrew,
Getting rid of the ratings system is worth all the headaches.
March 19, 2008 7:11 PM | Reply | Permalink
Okay, I'll stand corrected on that one.
March 19, 2008 7:31 PM | Reply | Permalink
I got zeroed out a couple of times for posts where I was in substantive disagreement with most of the TPM readers but I would be happy to have the system back because it gave us a method to hit someone for contentless posting without taking up anyone else's time. Even though the system was abused by me and others to idicate substantive disagreement, that aspect made it worthwhile.
March 20, 2008 9:00 AM | Reply | Permalink
Yeah, getting rid of the ratings was intentional. We felt it generated a lot more headaches and animosity in flaming and downrating wars than it accomplished in organizing the conversations and highlighting quality.
But we do think there should be a way 'flag' inappropriate posts. It wouldn't be for something you disagree with, or something just lame or frivolous but inappropriate language, abusive conduct, extreme trolling etc. that would be a big help for us in policing the site.
March 21, 2008 12:35 AM | Reply | Permalink
Two things. To solve the logging out problem, keep two tabs open for tpm. When you comment, go over to the "other" tpm tab and go t the main site. Log in there simply by clicking "log on" or go to your profile and log on there. That will decrease the losing of posts and the annoyance of signing in... as you'll get an automatic sign in.
Andrew, all of us know that much of this could have been prevented by pretesting the new software etc. Many of us would have been glad to assist. So it's hard for us to sympathize here, given the lack of pretesting prior to rolling out this frustrating software.
Finally, it's been one day short of 8 weeks.... since all of this suddenly happened. 8 weeks of our blogs in the "lost luggage dept." Honestly, it's hard to believe the story of what happened to them. In any case, to have not given us warning that they would disappear.... possibly forever, well that's nearly inexcusable.
You guys are looking for sympathy. But we're on the other end of this feeling very, very frustrated and far more helpless you folks who had some warning of this.
March 19, 2008 7:05 PM | Reply | Permalink
"First, there's the problem of folks being logged out mid-session, causing them to lose posts they're working on or longer comments."
How I avoid losing my posts. After I have typed them, I use the mouse "select all", then "copy" functions. If I lose what I have written, because I have been forced to log on again, then all I have to do is use the "paste" function, instead of having to start all over.
Perhaps someone else may have a better temporary work around than this, but at least it has saved me a lot of frustration, and retyping.
Now to select all and copy before I send this post.
Cheers.
March 19, 2008 7:06 PM | Reply | Permalink
Andrew,
A flagging system for abusive comments.
March 19, 2008 9:38 PM | Reply | Permalink
Andrew,
Thanks for your efforts and for keeping us up to date.
It seems odd to me that one may repeatedly click "recommend this", even on one's own post.
March 19, 2008 9:45 PM | Reply | Permalink
That will look like it works but if you reload the page it will actually only accept one recommend per user.
March 19, 2008 10:30 PM | Reply | Permalink
Oops. Thanks for the correction Andrew.
March 20, 2008 9:25 AM | Reply | Permalink
too cute! it looks like it works, but ...
lots of features here ... wait, you wanted continued patience; right?
bogus climbing recommend hit totals--swift! keeps the delinquents busy for awhile too, I guess ... giving those stinkers less time to write up stinky trolling rants etc ...
come back again real soon--don't feel you have to wait 8 weeks next time!
that's what I think of your snarky reply to my email regarding my latest problem ... yeah, that 'did you request it' line -- that was too cute too!
I think you were just joshing me.
March 20, 2008 9:55 AM | Reply | Permalink
Flagging system for SPAM and clearly commercial posts.... (ie - those damn Auto Loan companies infesting the site lately...)
March 19, 2008 9:46 PM | Reply | Permalink
Regarding possibly useful features: today someone posted about a recent troll eruption on the site. I was wondering whether it would be appropriate to publish every poster's IP address - for instance, after the 'posted by' at the bottom of the comments postings. This would be fairly easy to implement since you guys already log the IPs anyways (probably a simple database call). The point of such measure would be to foster transparency - it would also function as a strong incentive for posters to keep things civil (and yes, it would certainly deter coordinated trolling).
Best
March 20, 2008 3:47 AM | Reply | Permalink
Thanks for the update.
March 20, 2008 8:44 AM | Reply | Permalink
It's nice and, I think, appreciated that you try to be as responsive as possible while cleaning up the mess made by a bad decision (to convert from the old format to this one which sucks).
The situation still reminds me of working in an organization that decides to "improve" it's system for whatever (computers, phones, etc.) by installing one that doesn't do the job as well as the old one, but initially is billed as being a trade off wherein the new system does x, y, and z better so the loss of a, b, and c is somehow justified. Unfortunately, as is usually the case, these are just excuses justifying a mistake in the first place.
Now, you are stuck trying to fix an inferior system that simply isn't as good as what it replace at least from the perspective of the users. I still don't understand how or why anyone on your end thought this was going to improve anything at TPM. And I've never gotten a reply to my queries about that. C'est la vie.
Sometimes it's better simply to admit the bad decision was a bad one and go back to the old system and instead of junking it, improve that one which, with all it's flaws, was head and shoulders superior to the clunker that's being foist upon TPM staff and users alike.
Naturally, I know this is all an excercise in futility as it's clear TPM is as determined to make this inferior system work as Bush is to try and convince the world that we're all safer because of his bad decision to invade Iraq. Still, it helps to get it off one's chest.
March 20, 2008 9:34 AM | Reply | Permalink
Oleeb, this may sound good in principle. But with all due respect, you're either not paying attention to or not aware of the full context in which we made this decision. The new version of movable type had many more bugs than we were lead to believe. And that was a real nightmare for us and for our users. And for that we're very sorry. But we could not keep having different parts of our network on completely different publishing platforms. It wasnt workable in terms of cost, function, anything. So I do not blame you for your frustration. But your broader effort as social commentary doesn't square with the facts.
March 21, 2008 12:48 AM | Reply | Permalink
I'm from MO -- the 'Show Me' state ...
what we have now was 'planned' ... well, maybe not ...
in fact: let's hope not!
but do give us *warnings*; okay? timely warnings...
and in anxious anticipation, we'll be trying to prepare (gird?) ourselves in the meantime ...
ps I really did not appreciate the tone of your reply to my email about that latest fiasco I had with this mess ...
maybe you need more sleep ... no, not on the job ...
March 20, 2008 10:34 AM | Reply | Permalink
I don't know what Andrew's reply was like. If it was impatient or unkind, I apologize on our behalf. But you've also got to understand the position that Andrew especially is in. We run a series of sites that are far bigger in traffic terms than we have staff to manage them. The switchover to the different platform has been very unpleasant in many ways -- not least of which for us. Sometimes the best plans don't come off as expected, and the situation has been escalated all the way up to the executive level at Six Apart, the company that sells Movable Type. I completely understand the frustration. But Andrew is on the receiving end of a lot of very heated email, with complaints about problems that in most cases weren't of our doing and which it's very difficult for us to solve in the short term. So at the end of the day, we're human. And being on the receiving end of that much negative energy can lead to some testy responses. Like I said, we're committed to fixing all of this and we're starting finally to make some progress. But when you say this about Andrew's response please try to have some perspective of the situation he's in. This whole set up is run by Andrew, as the first line and then me overseeing. And that's only one part of his job at TPM.
March 21, 2008 12:41 AM | Reply | Permalink
Thank you, Josh. I can accept what you're telling me and *how* you're telling me. I will try (keep trying) to appreciate Andrew's position and his having such a heavy load, including having to deal with problems not of his making or under his direct control, etc.
It's even possible his tone wasn't quite so flip as it sounded to me at the time, but I became even more frustrated when I read his reply--which hardly helped!
Some of this must be related to the unexpected quirkiness of what the software is doing and how we, or I, feel when our requests for help seem to miss their mark as far as getting a response that relates to our experiences in particular because, again, it's difficult for anyone, including the 'sys op guy', to realize exactly what happened or why, thereby reducing his chances of being able to actually assist someone who already needed instructions which were entirely lacking anyway since it was impossible to foresee their need, etc etc ie, who knew? none of the above! hahaha and I'd been caught, by that time, in just too many endless loops of totally frustrating failure! so when the 'tech guy' himself seemed merely to perpetuate that pattern as well, I was basically dumbfounded until I got irritated ... tsktsk ... at least I'm not alone in this trial!!
but I do really look forward to having the kinks kicked out so that it all starts to purr along again ...
I also enjoyed those tidbits re ongoing back-story of this situation as well; jeeez, fascinating--but infuriating at times no doubt too!!!
and, your response really helped me*^! xox
March 21, 2008 4:09 AM | Reply | Permalink
A word or two a little more often would go a long way to keep frustration down.
March 20, 2008 11:23 AM | Reply | Permalink
we're going to try to do better on that count. Part of the problem is that we ourselves have been in the dark a good deal of the time. And we've been wary of making predictions and promises that we weren't sure were going to bear out. But we hear you on this front and we're going to try to do better in keeping the community informed.
March 21, 2008 12:44 AM | Reply | Permalink
You saying that you guys have been in the dark to some degree is in itself helpful I think. Part of the frustration I've felt and possibly others may have felt as well is the sense of being left swinging in the wind as it were, not knowing if our concerns were still on the to-do list there or not.
I've suspected that you guys may not have wanted to come right out and say that things are a little out of control, creates a bad image and all that. I actually think that in this medium, particularly, the important thing is to not try to appear as something you are not. Your forte is the intimacy of two way communication. When we think you're not hearing us it ain't good. As it turns out you're not "them" but more "us" after all, we're in the boat together as it were.
What you are selling is trust. You don't need to be perfect wizards at web tech to build trust. Trust is built when we think we are hearing each other.
OK 'nuff said I guess.
March 21, 2008 11:36 AM | Reply | Permalink
The problems I have recently had with TPM have been frustrating, and at the same time that the changes here have occurred, Firedoglake and Rick Perlsteins "Our Future" have changed their format to something less readable and less usable.
It's sort of like living in a comfortable small town that grows almost overnight into a much less comfortable big city.
Considering the fact that I have found those sites because our political system is broken and they offered some hope of fixing it, the frustrations seem to be caused by the fact that more and more people feel much as I do and are also moving towards the sites that I think are making a difference.
It's not comfortable, but on net things seem to be moving in the right direction. The discomforts appear to be caused by the symptoms of success.
Andrew, my thanks to you, Josh and the whole crew. Keep up the good work.
Now I'll just go join my aging curmudgeonly White West Highland terrier who hates the sounds of kid’s voices and thunder, and like him I'll grumble sourly at the discomfort while you guys are working hard at making my life more comfortable.
Growl. Grumble. Grrr.
March 20, 2008 12:07 PM | Reply | Permalink
Richardxx said:
For me, it was like living in a small town and suddenly waking up one morning in a new, different, and entirely disfunctional town!
But I am trying to be patient. It's lots harder being patient when there isn't much communication about what's going on, so this thread helps.
One other thing that neither you, Andrew, or Josh seems to have mentioned is we need a rich text editor, so that those of us who aren't HTML saavy can post without weird formatting errors. Even the ability to edit posts won't help if one is basically uncertain about how to format with HTML.
But there are good signs too. I was able to change my password, just very recently I seem to have to re-sign-in less often, and on my blog page, my own posts are now arranged in consecutive order with the most recent first (I can only access the first page of them though, so that area still needs a bit of work). Still, it's a big improvement. So thanks Andrew and Josh for the update and the continued efforts to get the site problems ironed out.
March 21, 2008 11:37 AM | Reply | Permalink
Thank you for the update, Andrew.
Good to hear you will be working on the "my conversations" page. That is something that would be much appreciated by me and, based on the public feedback I've seen, many others as well.
March 20, 2008 2:24 PM | Reply | Permalink
Yeah, once we get the basic functionality issues ironed out, this is our top priority.
March 21, 2008 12:43 AM | Reply | Permalink
Thank you, thank you, thank you. As one of your recent immigrants I understand both why there's suddenly so much interest here and sympathize with the adjustments it takes to catch up with the workload. I'm glad you're here, glad you let us participate, and look forward to being able to correct my stupid typos.
As for the editing function, I think a mandatory post preview step would be even better. Having to actually see your words can make you not only fix the typos and add some alliteration but see what they'll look like outside of the little teeny box.
March 20, 2008 9:28 PM | Reply | Permalink
This software's installation glitch where author names are missing from many individual Reader Blog pages,
which noted by me, and acknowledged by you, probably about a month ago now,
STILL HAS NOT BEEN FIXED,
and still causes confusion daily; example:
If such problem is to remain over a long span of time, it is a usual courtesy to the audience to put a prominent annoucement, rather than relying on good samaritan users to explain things to other users over and over and over.
March 20, 2008 10:42 PM | Reply | Permalink
THIS IS EXCELLENT NEWS!!! FOR TPM BLOGGERS!!!
March 21, 2008 2:09 AM | Reply | Permalink
Hi Andrew, Josh and All,
(Since this is an old thread (that I just found), I'll send this to the e-mail address as well.)
I appreciate your efforts.
I have a few of suggestions, which may or may not be easy to implement.
1) Permit editing of posts, but have a link at the bottom that calls up the previous version. That lets the reader see what was changed (and keeps authors from revising history).
2) Have a default Preview view before a new post goes in the queue. If that's not possible, give the poster the option of putting the post in a Test forum. (Its not immediately clear what HTML tags are permitted, etc., so it's not easy for newbies to know what the post will look like).
3) Make the "Recent Reader Posts" queue longer (double?). As the Cafe gets more popular, this list will be scrolled faster. As it stands, readers of Election Central often won't get a chance to see many posts if they only stop by a few times a day. Clicking on "All Readers Posts" isn't a full solution because that link brings up all the posts in reverse chronological order - meaning you have to sort through them again, and if you get to the bottom and click "Previous Post" you get the full list from the current time and have to sort through them all again. Adding a dedicated column for reader posts and recommended posts might be a solution, but I know that would be a very big change.
I'll quit now. Thanks again.
March 28, 2008 7:39 PM | Reply | Permalink