Against linguistic determinism

In several places this week there have been reports about the research of Dr Peter Gordon of Columbia University on an Amazonian tribe which has an extremely limited sense of number and ability to count - this correlated with the fact that they have no number words beyond their terms for 'one' and 'two'.

What interests me in the reports are claims of the following kind:

Language moulds our thoughts so much that we cannot conceptualise ideas for which we do not have words...
.....
Dr Gordon's work supports findings made in the 1930s by Benjamin Lee Whorf, a linguist who argued that language can determine the nature and content of thought.

Or, again, see the reference here to 'linguistic determinism', 'the controversial hypothesis that the language available to humans defines our thoughts' - though (from the same source) it seems that Peter Gordon does not himself subscribe to this hypothesis. For he's quoted as saying:

There are certainly things that we can think about that we cannot talk about.

Gordon only argues from this research that for numbers 'a limitation in language affects cognition'.

In any case, it continues to perplex me that anyone at all does subscribe to the hypothesis of linguistic determinism. Unless one makes the hypothesis true in a merely tautological sense, so that we don't allow as counting for 'thought' anything which can't be formulated in words, it seems evident that there are both pre- and sub-linguistic forms of perception, awareness, consciousness and so forth. Think about how you know - exactly - where to place your hand in catching a ball; or, more to the point, how you discriminate between the elementary sounds of which spoken words are made up, so that you can tell what words are being uttered, and thereby gain access to language and the concepts which language makes available to you. This ability so to discriminate those elementary sounds doesn't seem to rest on a linguistically-based conceptual apparatus; and if it did, it would be hard to understand how children come to acquire language in the first place.

So I'm puzzled by those (linguists, postmodernists, discourse theorists, etc.) who argue that everything, for us, is through language. I spoke of this puzzlement once before here, in a more frivolous way than I'm doing now. Is there anyone out there who's properly versed in this area of philosophy (as I myself am not), and could help to clear up my puzzlement?