It's your job to be honest, otherwise peer review is pretty pointless. If you are honest and fair, and the outcome is negative, then a grown-up should be able to handle it.
... I've just been asked to review something that is far outside my area of expertise. I have no idea how they came up with my name, and probably shouldn't have agreed to it, but I suppose I'll just have to do my best.
If this is a journal that's fairly specific in its focus, and not very relevant for you, it is OK to politely decline. If it is a broad journal, then often editors will pick one or two reviewers who are specialists in the paper's focus, and one who represents the "general readership". The latter often give reviews that help the paper become more accessible and easily cited, which benefits the author and the journal...
This. It's ok to read a submission and say, "You know, I am not right for this after all. Please find someone else". However, you should do that within a day or so of getting the article, not waiting the full allotted time and then saying it.
I know I could have declined it - it was a bit of a panic decision. I was on vacation, sitting in an internet cafe after having not checked email in days, and saw both the original email and the "reminder, late response to invitation" email. I glanced at the abstract, thought hmmm, looks interesting, what the hell, and clicked accept. I've also never reviewed for this journal before, and thought it'd be nice to add another to my repertoire.
The "general readership" aspect is a good point, and a good angle to approach the review from, thanks. It is a broad journal, and the paper is very interdisciplinary, so maybe they are looking for people from both sides. I do know the geographical area very well, and have some expertise on one of the arguments, so it's not completely out to lunch. Maybe I'm meant to be the reality check for that line of argument, even though the broader question is not really my field.
I do always find it interesting to try to figure out how I got landed with certain papers. Some are obvious - I was cited, or I know the editor, or it's firmly in my area of expertise. This one is a bit of a puzzle...