why should it be only I who has to suffer the consequence.
Well, you are the one that received the "extra" money, you are the one that is benefiting from the extra money. If the escrow company was still holding the funds in their account, then clearly they would need to pay up -- but since they handed over everything to you, you had the money, so that makes you responsible.
You are right, somebody dropped the ball on this -- but that doesn't change the fact that the buyer is owed a certain amount of money, and you were paid that certain amount of money. This situation is somewhat analagous to paying a bill using your bank's online bill paying -- if the bank screws up and doesn't send in a payment, that doesn't relieve you of the obligation to pay the creditor when they come asking why you didn't pay them. Even if the buyer did decide to go after the escrow company, and even if the escrow company did have to cough up cash, the escrow company would then come after you.
Maybe you can go back to the escrow company and/or your agent and get them to reimburse you some of their fees for your trouble and inconvenience, but the bottom line is that you are responsible to pay the buyer what he is owed under contract.
I no longer have the funds to give the buyers, “you cannot squeeze blood from a turnip.”
You may not have cash, but you certainly now have an asset that a lien could be recorded against. You are far from "judgment proof."
It would cost them so much more to take me to court than the money in question.
Why is this? How much to you owe them? If it's within the small claims limit, then it would be easy. If it's more than that, yeah, they'll have to hire a lawyer -- but they might just do that. Then what are you going to do?
What you should do is figure out some way to pay the buyer. Maybe they will accept payments, maybe you can take out a home equity line of credit, something -- but you are not going to be able to duck responsibility by claiming you don't have the money. If they get a judgment against you, they will put a lien on your house, and it will remain there, collecting interest, until one day you decide to sell or maybe even refinance -- then it will get paid.