Tuesday, May 3, 2016

Issues with a "Survey"

So I checked out the following HCPSS post today, "Board Seeks Public Input for School System Budget Cuts."  I am providing no link as I am not encouraging anyone to take this "survey."

I still don't know precisely who crafted the instrument or if they have a background in public opinion research.  A few elements struck me as off.  Let's look at what is posted:

Where to begin?  At least the sponsor of the study is identified.  That is nice.

Of course this "tool" is not being fielded using a randomized sampling method, it will not yield statistically significant findings that are projectable to a larger population.  It is completely based on self-selection. So why bother calling it a survey?

And FYI, COB means different things to different people.  5 pm? 6 pm? 8 pm?  Eastern time?  Who knows?

Beyond that, the language is leading.  They shouldn't be patting themselves on the back in the introductory paragraph, as it injects bias.  Even a subtle shift would have helped, something to the effect of, "In accordance with HCPSS' commitment to transparency in its budget process, the Board of Education has developed, etc..."  Frankly, the whole transparency claim here is extraneous.  It reads like posturing, not the beginning of a research instrument.

Then it goes on to frame the two proposals.  It is not a bad idea to provide some context.  That said, this introduction appears to praise the BOE request, placing it in the most favorable light, while essentially saying the County Executive's plan would gut education funding in our County.

Careful, and sloppy, readers of this blog know that this author is not a Kittleman supporter.  But come on, try to present a more even-handed case for the CE's proposal.  Say how it represents $18.1 million more in funding compared to the previous year.  You could even ding his plan (factually) for being just slightly above the state's minimum funding requirements, but just saying how it is $50.4 million in proposed cuts is sticking one's thumb on the scale.

Is there a guarantee of respondent anonymity?  No. Which should be disconcerting to many.  In fact, there is this lovely part of the tool, before the substantive questions appear:

So this makes me wonder.  Will folks at HCPSS see my answers attached to my name, address, etc... or are they going to disaggregate my identifying information from my responses? Who all is going to see this?  Is the data going to be collected/analyzed by a third-party? If so, who?  Will the Board of Education know that I responded...and how?

I didn't go any further as I did not want to input my personal information.  Frankly, I think the set-up they have is a tremendous disincentive to participate. Perhaps that is what they want, I don't know.

Gathering insights from the community is fine (and appropriate), but trying to use a tool that implies a certain measure of scientific rigor, without actually following proper practices regarding wording and data collection, strikes me as problematic.

Stay tuned, as more will follow.
 


3 comments:

  1. I wanted to see what was there so I just popped in animal names and for my name and address. Phone # was 000000000. I got to look at the rubbish. It was rubbish. Not a single cut from Central Office staff. They have all the minor stuff at the beginning and then hit you with teacher cuts, furloughs, COLA etc at the end. It calculates your cuts so that you can see how close you can get to the $50.4 million. Then you can submit. Gee, I wonder where all the savings went from the big retirement exodus of last year? Thought that was supposed to save the county millions of dollars over the next few years ? You should fill in some bogus demographic info and take a look. It's just like every other thing they put out there....smoke and mirrors!

    ReplyDelete
  2. I wanted to see what was there so I just popped in animal names and for my name and address. Phone # was 000000000. I got to look at the rubbish. It was rubbish. Not a single cut from Central Office staff. They have all the minor stuff at the beginning and then hit you with teacher cuts, furloughs, COLA etc at the end. It calculates your cuts so that you can see how close you can get to the $50.4 million. Then you can submit. Gee, I wonder where all the savings went from the big retirement exodus of last year? Thought that was supposed to save the county millions of dollars over the next few years ? You should fill in some bogus demographic info and take a look. It's just like every other thing they put out there....smoke and mirrors!

    ReplyDelete
  3. Regarding early retirement watch the BOE hearing. They were scolded for now saying they need more money to hire experienced teachers when in past years they've said hiring new staff members will generate savings (as opposed to cost more) since entrance salaries would be lower and the buyout would save money.

    ReplyDelete