As a writer, I tend to create stories that appeal to me, which is great if I'm buying my own books. The trick is, to find an audience who loves the same stuff I do. How does one do that? Picking the right publisher is a great start. Pick a publisher that sells books you like to read. Buy some of their books in different genres to get a feel for what they sell.
The first book you write for that publisher will be what your audience expects. ESPECIALLY if you write something with a speciality audiance. Menage. Slash. Manga. Shunga. Fem dom. It's much easier to cross genre if you start simple. M/F audiences will cross over easier than say ménage readers will cross to straight M/F books. Likewise, if your publisher specializes in a genre like science fiction or paranormal, it will be easier to cross over, more than if your publisher specializes in historical or sweet romance. That's an extreme, but you see what I mean.
Likewise, IR publishers will publish ménage, but it's not necessarily what their readers expect. You may contract the book, but don't expect to set any sales records.
Bottom line? Write what you love —it's what makes a good writer a great one—but research the market. Find a publisher that suits you and has a good track record of selling books in your preferred genre. Once you’re in, you'll be on your way to building a dedicated readership.