Measuring Healer Performance

I started to write a longish post about using stats – WWS in particular – to evaluate your healing. I realized, however, that we don’t really have a good measure. In a lot of ways, most healers are stuck at the same stage as a lot of DPSers who don’t study — the only thing that is measured and considered is gross results. For them, “did I kill the mob without dying” is sufficient. For us, “Did anyone die? No? Then that’s the only measure that matters.” Yes, in the end, it is. And yet, what we want to do is to look at our performance so that we’re READY in the event we hit, well, the nasty pulls and Big Bosses.

I think that there isn’t yet a tool out there that will really help us, but I might be wrong. I think that it might be possible to develop such an analysis, but I might be wrong. I can’t know whether I’m right or wrong about either of these possibilities until I have decided what I – and I hope WE – need. Therefore I’m going to ask the Egotistical Priest’s question again, only this time intentionally searching for quantifiable measures that can be used to obtain the qualitative truth she identified. It’s going to be a long slog, so take a moment to get comfortable and, hopefully, be ready to add your insights.

Let me begin with the absolute with which everyone agrees. A GREAT healer is one who never lets a party member die. Which points out a major difficulty of our measures — compared to all other roles, the healer role is massively dependent on the performance of others. Get an extraordinarily geared tank and you can practically sleep. Get a warlock and a mage competing for top of the damage meters without concern for the tank’s aggro and you can see death after death even before the first mob. Thus I think we have our first caveat — our first uncontrollable conditional against which all quantifiable measures must be considered: the group. I’ll probably be returning to that element. But let’s get back to quantifiable elements. Specific in this case: Deaths.

When we look at player deaths, we need to identify not why they died, but whether we could have saved them. Which basically means we need to know two things: How long did it take them to die, and what were we doing in that time?

I have identified my first stats to be parsed. I want a tool that will show people’s health – as a percentage, I think – over time in chart form. And on it, I want all my healing casts shown. Probably it should show ALL heals, identified, so I can pick up on when my PoM (or other classes’ unattributed healing) is the cause. hmmm, more complex than that – I need it to show things like shield, too.) Better put on a “Died” indicator so I can tell fairly easily whether they died and were battle rezzed or if it was a feign death or 1% situation.

Another major issue is mana management. I think I should have another line on that same chart – extractable as a separate, perhaps – that shows my mana percentage. Most of the spells I cast are there already, I need to know when and how I’m gaining mana, too.

Huh. I’ve just identified a ‘better than good’ element for measure. Mana management. Assuming you don’t let people die, how ready are you to keep going? Yes, overheals play a part here, but far from the sole element. PoM or a PoH that brings 4 people up from 50% and is 100% overheal on the fifth are examples of where we just don’t care. Though… the first chart will show some of this, too.

Which means one of the steps of getting “better” is learning to be not-too-quick on the trigger. You want to maintain player’s health at levels high enough that they won’t die to a heavy attack before you can heal, but other than that… let them bandage. And again, it’s quantifiable. Not just overheals, but how often do you heal to 100%? 90%? 80%?

In fact, let’s add this as a statistical, quantifiable measure of worth. What proportion of your heals go off at under 20% of the target’s health (bad), and what proportion take your target over 90% (possible mana waste)? (and for comment from you all — is 80% a better measure?)

Statistical measures: party health % and healer mana status at end of battle. “Perfect” is both at 100%. Hmm. Add tank health % to that, I think. I suspect there’s a valid ratio or proportion in there, and since dead tank is absolute bad it’s probably better expressed somewhat better as health remaining per mana remaining — but lom with 100% health is also bad. I’m going to guess this is a multiplicative value – that is, 100%*100%=100% is excellent, while 1%*100% = 1% is, well, bad. I suspect there’s a better weighting but the H*M value has good potential and is relatively easy to calculate. I’m absolutely certain a tool can be made, and 99% certain one doesn’t already exist.

I think… I think I want another ‘waste’ calculator as well, though this one is potentially going to be used to beat other players about the head and shoulders. I want a tool that measures total healing delivered over the target’s max health. In most circumstances the top number should belong to the tank. If another player is ‘too high’, we may be wasting heals. Or they may be abusing the threatmeter. Or we may be doing the right thing as the mage is AoE pulling a bunch of adds. Still, I think the distribution curve will give us a better tool for analyzing our performance on a quantifiable basis — especially if it can be made to count overheals by target.

Yes. Yes, I want that too for statistics. Which players did I overheal and with what spells? If it’s the tank, I really don’t care. If it’s due to group heals (as already mentioned earlier) saving part of the group while wasted on the rest, I don’t really care. If I’m doing repetitive overheals on the hunter… am I getting caught by his feign death? Panicked by how fast he drops on the rare situations he DOES take damage? Need to downrank for his heal? I can’t even ask the questions if I don’t realize HE is my overheal target.

So, let’s recapitulate and summarize, shall we? The stats and reports that will help us become better healers include:

H*M – percent of health remaining * percent of mana remaining at the end of a fight, with an improved version being the (as yet undetermined) weighting of the elements. H*M is needed for both party and for the tank – the latter meaning I should be able to apply it against single members of the party.

Differentiation of overhealing by both spell AND target.

Related – proportion of heals that kick in when target is under 20%, and proportion that heal to over 90% (with discussion as to whether 80% might be a better target).

GRAPHIC DISPLAY of players’ health over time of the battle plus my mana, all on a percentile chart, with events indicated – healing spells (preferably with a cast-time bar so I can see I didn’t heal X because I was still casting GH on the tank) and events (bandages, pots, etc) and mana events ( at least restorations, possibly mana drains).

So what separates a good priest (qualitative value per Ego’s post and the discussion) from a better priest (good plus)? Or maybe… what are the quantitative factors that help you become a good priest? How effectively and efficiently do you heal, I think. Timing and mana efficiency. No deaths, and enough mana at the end that if something DOES go wrong (which in an instance happens sooner or later), you’re ready to cope.

OK, I’ve rambled. Thought – not just on what I’ve written but your own quantitative factors? Remember, I agree completely with Ego that a good priest is a measure of quality – of overall performance. This is an attempt to identify what measurable factors matter. Because if they’re measurable, we can work to improve them – but only if they matter. Your turn.

About these ads

~ by Kirk on September 28, 2007.

23 Responses to “Measuring Healer Performance”

  1. You know, that’s a good point. I’ve been looking at this from a “What can current meters tell us” instead of an “if I could use any battle data, what would I want to measure” angle.

    And sometimes building wishlists like that is a good idea. I may take your article here and create my own, though you may have covered everything I can think of. =]

  2. “proportion of heals that kick in when target is under 20%” – I think this one only matters on the tank. If the tank gets that low too often you might need to look at some things, but 20% of a mages health… it’s just too easy to get there with only 7-8k health.

    I really like the idea of knowing WHO is getting the most overhealing. The lock is going to be a prime target b/c of Renew’s after Life Tapping… but still usefull info.

    The % mana vs. % health remaining would be dependant on the build of the priest. A priest with a more spirit heavy build will always have more mana left over. I end a lot of Kara fights with lots of mana left over. The other healers were giving me funny looks (metaphorically speaking) because they thought I wasn’t doing anything. I had to explain that I get a lot of mana regen while casting (soon to be more, thank-you patch 2.3), I time my use of pots and the Shadowfiend very well and I’d like to think my self an expert and dancing the 5 second waltz. I’ve run with other priests that just don’t have the spirit that I do and can’t last as long. I usually end the fight with more mana than our crit specced pally with 27% crit chance. But now that I’m thinking about it… I guess that’s the point… how well do you use your mana resources?

    The last thing I would consider is assigned healing targets. Did Healer X’s target die because he was healing Healer Y’s target? There’s no way to put that into a combat log… but it’s quite meaningful info for me. My tank did a lot of dying in Kara last night and he kept saying things in the officer chat (I’m the only healer that’s an officer) like “Why can’t you guys keep me up?” (which is his way of saying “WTF is wrong with you tonight?”) I finally had to tell him that he’s not my target… the other healer is in charge of his health and I only throw heals his way (other than a POM and Renew) when things are getting bad which last night was too late. This other healer is too reactive, not pro-active, and his gear just isn’t as good as mine (which is another rant I may write about today… how do you decide who the MT healer should be?) The third healer in Kara last night was also spending entirely too much time on my targets in the Curator fight (we had 2 rogues and the OT on the flares so they were continuously chaining the lightning, and were therefore my dedicated targets), to the point that I had to start healing HIS targets. Kind of defeats the purpose for assigning targets.

    Good thoughts here, I look forward to seeing what else comes out of this disucssion.

  3. @Ego – That’s what I discovered when I really LOOKED at explaining how I use WWS for priests – a post that’s probably going to die unfinished in drafts, but… And please, yes. The more of us who do this, the more likely someone with real programming (vs scripts for macros) says, “Hey, I can do that…”

    @Galadria – good point. I was tempted to put in a floor value instead, but that floor is variable depending on level and players and… I’m open for suggestions. And know that like any stat report it can be misused.

    Yeah, mana management is the point. When I really get to looking, our Big Deal is the five second waltz and mp5. Which means our Big Deal is mana management – it is what allows us to be effective healers. How much mana are we spending per second, including mana recovery (negative expenditures)? Second, of course, to “nobody dies”.

    Agreed, an ideal element of all the tools is to ability to subgroup. That is, not just single target and “everybody” but “choosing these people from the party as a whole”.

  4. Let me throw out a few things here, almost from an outsider’s viewpoint (I simply don’t have the expertise or the experience you three have, but I do have a broad perspective, with 3 classes at 69 or 70):

    First, Gala says she has lots of mana left at the end of fights. That’s a Good Thing. But…is everyone else at 100% health? Should they be? After all, mana is a renewable resource; bandages aren’t (within the context of a given instance, of course).

    I think back to something BRK says regarding Hunters: Ideally, as a hunter, I want my mana to drop to zero at exactly the same time the boss’s health does. (As a rogue, I don’t want to be standing there with 5 combo points after the boss dies, either.)

    It comes down to (as we ALL know), resource management in the context of situational awareness. I know when I’m healing in an instance, I feel pretty good about myself if everyone’s at or near 100% health and I’m almost topped off on mana. But if everyone’s at full health and my mana is in the red zone (i.e., <15% or so), have I done as good a job? Or how much did I overheal? Now that the fight is over, does it even matter? (I think it does…because now the group needs to wait for me to drink; of course, I’m probably not the only one.)

    I also believe (and I think Kirk and/or Ego touched on this) that while not having any deaths is the most desirable outcome, just because someone died, it’s not automatically my fault. If the mage moved into aggro range of the next pull, and ends up dying while I’m healing the hell out of the tank to avoid a wipe, how is her death my fault?

    Sorry for rambling–this is a great discussion, but what I’m trying to get at is I think there are more intangibles involved with measuring healer effectiveness than in measuring DPS. Not that I wouldn’t want a tool along the lines of what Kirk’s looking for–I just see a lot of practical issues that may be a lot more difficult to overcome.

    HOWEVER…let’s not focus on why we can’t have such a tool, but rather, how can we get it to work for us? :)

  5. @ Kirk: Just reread your last comment, and absorbed something I should have the first time around.

    Mana management IS what it’s all about: After all, if we had unlimited mana, we be spamming GH7 every which way, with the occasional POM for good measure. So maybe what we need is something more than a simple 5-second counter, just to help us with mana management.

    But Gala makes a good point too: If we can’t tell from our HealStats program who we were SUPPOSED to be healing, how do we know how well we dead? We can probably make inferences based on who we DID heal…but how good are those guesses?

    I really, REALLY wish I were a programmer…

  6. Kestrel, right. see, for killteam BRK’s exactly right – zero mana reached exactly as the targets hit zero health means you burned mana RIGHT. For the killteam mana not burned is wasted – because using mana harder usually means you killed the mob faster, which is good in so many ways.

    The reason this isn’t so for the healer is, well, because in a “perfect” fight you do almost no healing at all. Just a renew every so often on the tank. Your contribution to the fight is allowing everyone to fight longer if it’s necessary.

    You are a marathoner working to help the sprinters do another lap, if you’ll pardon the stretched analogy.

  7. Found a partial, asking for a few lookers.

    Get Recount. It allows graphic display, separated by individual, of heals (who you healed with what), last few seconds of death (and log of what happened), mana regen sources and amounts…

    Lots of things on the want list aren’t there, and it’s not really in the form I want, and it can’t be saved by encounter for later review, but… it’s still got one or two useful items.

  8. I have Recount…I’ll have to take a look at it from last night’s MT run, just to see what it tells me. As I was reading your original article, I was thinking about it. Didn’t mention it because I thought you’d probably already considered it.

    I haven’t really used it much, but it IS pretty! Those graphs are VERY cool.

  9. Yeah. It also eats a lot of cycles and memory.

    See, I play on a laptop – a Dell Latitude 110L, if that helps peg my situation for you. Which means I’m REAL conscious of where my cycles are going. Which in turn explains one big reason for preferring WWS to SWS – out-of-game vs in-game analysis.

    But those heal charts have potential…

  10. I have another stat for consideration – no, make it two.

    1) average health of the party over the fight.

    2) Proportion of time spent in and out of the 5 second rule.

  11. [...] hand, measuring a healer’s performance is a bit more ethereal, as discussed in depth today on Priestly Endeavors. Of course, you can watch the healing meters, but they don’t tell the entire story of a good [...]

  12. [...] hand, measuring a healer’s performance is a bit more ethereal, as discussed in depth today on Priestly Endeavors. Of course, you can watch the healing meters, but they don’t tell the entire story of a good [...]

  13. Grats on the WoW Insider plug. Too bad some of the comments are … how to put this nicely?

    Too bad some of the comments are there. (How’s that?)

  14. Nice post Kirk, you’ve obviously given this a lot of thought. I can’t help but wonder if you’re overcomplicating things a little bit though.

    Since the problem you’re addressing is not necessarily that the healing meters (as they exist today) can be inaccurate but that a simple healing total is insufficient to reflect healing skill or efficiency I’ll assume for a second that we have a magical healing meter than can give a completely accurate total for each healer.

    I would argue that most of your metrics are rolled into that healing total, at least on fights where it matters. If the fight warrants efficiency then the inefficient healer’s numbers are going to flag as the fight progresses and he runs out of mana. If the fight warrants burst healing and the healer’s timing is off then more of his healing will be registered as overhealing. If that healer was assigned a target then his healing on that particular target should be considered (or at least more heavily weighted), not his total raid healing.

    I also wonder about a situation where the amount of healing necessary exceeds what the healers can output efficiently, forcing at least one healer to switch to a much more inefficient heal rotation to account for the extra damage. This might blow their mana pool and cause them to appear inefficient in any isolated measurement compared to the healers maintaining a more sustainable hps, yet their transition was necessary and helped the raid by allowing the other healers to conserve mana.

  15. @Michael, good points.

    One thing to keep in mind is that I don’t want a tool that perfectly measures a “good healer”. As simple comparison, consider damage meters. They can be abused. But even when they’re NOT abused they can be misleading due to the specific circumstances.

    What I want to determine are: what are the bits of information that GENERALLY tell you if a healer is doing a good job or not? From that I’ll probably do another post asking folk to decide whether they think it’s (again generally) always, sometimes, or rarely useful. With THAT, I hope to go hat in hand to some programmers and ask them to develop simple tools to give us what we’re asking to get.

    So that as healers we can look at these, take into account the peculiarities of the event being measured, and see what we might do to be better healers.

    And due to the nature of healing, we cannot substitute heals for damage in all the existing analysis tools and get something that works for us.

  16. @Kestrel, thank you. And as for the readers, it’s apparent many didn’t jump over here but made their answers based upon what Elizabeth wrote.

    For what it’s worth, I actually found that what’s there so far is significantly more civilized than a typical forum squabble – or the stuff that I see striking some of the WowInsider writer’s pages.

    Dear WowInsider Reader. Welcome. If you find something of interest, I am glad. If you can pass on your own knowledge and view points, I am glad. Please realize I mean what it says up there in the “about” in the corner – up there at the top right.

    Also, be aware I meant it when I wrote This Article.

    Welcome.

  17. Get Recount, it’s a ace2 damage meter that also tracks a ton of other stuff. It’s quite as in depth as you are wanting, but pretty damn close.

  18. @Teysa, Recount’s already been noted. It isn’t, actually, pretty damn close. It’s ok, and it’s got some, but it doesn’t give some of the things I consider most important.

    It’s also got another two problems to my point of view. First – and a consequence of how MUCH it can do – it’s a cycle and memory hog. I could live with that, hoping for an eventual upgrade that let me turn off the portions I didn’t need – if it weren’t for what to me is the largest flaw (also existent in SWS and a couple of others) — the “All or Nothing” data. If I do two runs of something, I can either have data for both combined, or I can have only the last run. There’s no way to save and review later.

    That happens to be a huge reason for my liking of WWS. Save for later review and comparison.

    But again, there’s no mana out measure. There’s no data line that lets me realize I lost X because I was casting on Y. There’s no measure of how long I was casting, or how long I was in or out of the 5sr. There IS a breakdown of who received what heals and a number of related issues which are, in fact, useful. So… very good start, far from “pretty damn close.”

  19. I’ve been spending a lot of quality time with combat logs recently (developing these kinds of tools for measuring tank performance). The main problem is that you can’t calculate health or mana %s from the combat logs. Tools that run in game, like Recount, know how much everyone’s total health and mana are, but none of this data is reflected in the logs.

  20. So wouldn’t the solution be an addon that ran during the fights, which added health/mana/other data points to the comabat log (or its own log), and which also had an offline component that read the data and translated it into the proper break down similar to WWS. I think from doing some addon programming, that addons cannot write to text files outside of the game… except for at logoff…so that maybe a limitation you would have to work around. I’m not sure if you can add lines to the combat log and then have that logged out of game, but there should be some way to do this.

    Kirk, great blog…I am a new 70 pally healer, but I find the general articles and healing discussion very helpful.

  21. [...] hand, measuring a healer’s performance is a bit more ethereal, as discussed in depth today on Priestly Endeavors. Of course, you can watch the healing meters, but they don’t tell the entire story of a good [...]

  22. @Asu, actually, as near as I can tell recount doesn’t tell the total mana available (but I may have missed that). That said, that’s exactly my main issue with the program.

    @Leukos, yes. But I want to get the horse before I build the cart. Specifically, before I just go dumping data I want to figure out WHAT data is needed and WHY. Thus the discussion of the underlying metrics of what USUALLY makes a good healer so good.

  23. [...] a year and a half ago, Priestly Endeavors did a post about measuring healer performance. This is a great post, and I encourage everyone to read it.  Perhaps it is time to reflect on [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
Follow

Get every new post delivered to your Inbox.

%d bloggers like this: