" Doing something different with a pop song, breaking the convention of verses and choruses– it's something we always keep in mind when we're recording,” says bassist Tom Warmsley, who founded Temples with singer/guitarist James Bagshaw in the summer of 2012.

Looking like a West Coast psych band, Temples bear all the hallmarks of cosmic travellers. There's the band name, for starters, there are track titles that sound like JG Ballard novels (Prisms, The Golden Throne, Sun Structures) and there's the fact that they take incense sticks on the road with them. But if all of the above suggests Temples are backwards-lookin, think again. The Kettering four-piece's music is a mix of scuzzy glam stomp, dreamy, 12-string-drenched folk-rock, droning psych and more – all given a 2013 spin. Retro is a dirty word.

"Psychedelic music has always been forward thinking," says Tom. "|t's so easy to fall into that kind of revival band thing, but our aim is to reference these things and bring something completely new to it. A song like Sun Structures talks about something contemporary using old imagery and eastern religion. It isn't a fictional work that people won't be able to find anything in. And the fidelity of the sound alone says we're doing something that couldn't really be done before."

Temples did, in fact, begin with a mutual love of music and mysticism. The four bonded over the writing of Aldous Huxley and Timothy Leary, the films of Kenneth Anger, and the music of The Byrds rather than The Beatles ("They're hazier and more interesting," reasons Tom. "The Beatles give too much away; the Byrds make you work hard. It's that quest to learn more and discover more.").

Arriving back in Northamptonshire from their respective university cities, the band members' far-out pursuits provided escape from life in a grey, English market town. "We were writing, recording, looking for work, not really knowing what to do," says Tom. And the relative lack of excitement gives them more time to focus on the music. "It's such an odd town really for the arts, because there's nothing here," says James. "There are artists in Kettering, but there's no outlet for them."

Made in Kettering, Temples's first recording arrived in one technicolour burst in July 2012. The band recorded Shelter Song, put it on the internet, and set into motion an unstoppable chain of events that eventually saw them signed to Heavenly Recordings. That same track, a bouncy, psychedelic romp, was issued as their debut single – and quickly became as sought after as any of the vintage vinyl they individually lusted over. The 7" now regularly fetches upwards of £100 on eBay.

One of the more surprising things about the track is the fact that, like all of Temples' music, it was recorded at home, in the box-room of James's house in Kettering, an end terrace with a blessedly forgiving neighbour. "I'm always apologising to him for the noise, but he says, 'It's not noise, it's music,'" says James. The band aim for Jack Nietzche production on a DIY budget – and succeed. "It's similar to Joe Meek – he used to record vocals in his bathroom in his flat on Holloway Road," says Tom. "The way I see it, there aren't any limitations any more," says James. "If you know what you want to achieve, there's always a way around it. With technology today there's ways of emulating things that would have cost an arm and a leg years ago."

The response to Shelter Song confirmed this. But there was a problem with the wave of interest it set in motion. The band were, technically, not yet a band, and they were being offered gigs before they even had a full line-up in place. So James and Tom recruited drummer Sam Toms and keyboard player Adam Smith, Kettering men both, and worked on making the intangible tangible. Quickly, they became an accomplished live prospect, playing gigs up and down the UK, hitting the festival circuit and even sharing a bill with The Rolling Stones at London's Hyde Park in July 2013. For Record Store Day this year, they joined their Heavenly stable-mates for a memorable trip across the channel to Paris. Toy, Stealing Sheep and Charlie Boyer and The Voyeurs were on board. "It was amazing. It was like a school trip of a different kind," says James.

As well as The Stones, the band have caught the attention of a number of music's biggest names. Johnny Marr has declared himself a fan ("And he's obviously a huge ambassador of the 12-string, so it's nice he's picked up on us," says Tom), Robert Wyatt professed his interest in a letter to the band's management, Suede invited them on tour and Noel Gallagher came to see them perform in London. Some pressure for a new band, no? "Not really," says James. "I'm quite confident about it – there's not going to be any filler and no track's going to sound similar to the next.”

Right now, the focus is on knuckling down and polishing off that debut album. Mixed by Claudius Mittendorfer (previous credits include The Mars Volta, Muse & Franz Ferdinand) in New York, it is nearly complete and set for release early in the new year. The band have high hopes for 2014. "I'm looking forward to people hearing some of the songs we've not played live," says Tom. "Move With The Season is covered in 12-string but with this almost '90s-like looped beat in the background going on, and three-part harmonies on a huge scale. It's The Byrds via baggy."

It's a record that's destined to set out the band's stall as Britain's premier retro-futurists, with influences ranging from '60s psychedelia to Motown, glam, Krautrock and baggy, all viewed through a very modern kaleidoscope – and always keeping the song at the heart of it all. "We still want songs to be songs.” The key, says James, is innovation. "We never want to re-do the same thing, use the same formula as a previous song. We're always looking hard to better ourselves."