As for the Joyce comparisons, I guess two most prominent ones are these.
Allison Stewart writes in The Washington Post:
Newsom's albums are arcane and lovely, made to be pored over and interpreted. "Have One on Me" is her magnum opus, a three-disc set being likened to a freak-folk "Sandinista," though it feels more like a musical "Ulysses."
Randall Roberts describes the Sapokanikan video In Los Angeles Times this way:
As she moves, Newsom stares into the camera and offers a vivid recounting of James Joyce’s “Finnegans Wake.”
As for me, one of the most interesting ones, though, is this one by Dave Park from Prefix, which mentions some early Joyce as well:
People have said this will be Newsom's most accessible album, as if the wonderful squeak of her former voice (the singer developed vocal nodes in spring of 2009, and after two months of near silence her voice has changed considerably) was the only thing keeping people away from her music. That's a little bit like saying Ulysses is more accessible than Dubliners because it has fewer characters.
All these comparisons, though, are kind of superficial because closer study helps to discover some really striking similarities.
Structurally Joanna’s four LPs are absolutely parallel to four Joyce’s main works. Namely:
1. Milk-Eyed Mender – Dubliners
Both are pretty much collections of short stories.
2. Ys – A Portrait of the Artist as a Young Man
5 songs in the album and 5 chapters in the novel.
Both works are partly (or highly) autobiographical works.
3. Have One On Me – Ulysses
18 songs divided into 3 parts in the album and 18 chapters divided into 3 parts in the novel.
4. Divers – Finnegans Wake
A ring composition is used in both works.
The greatest moment of this parallel comes at the end of Time As a Symptom with its pulses of two-syllable word structures/images (including the quotation from Joyce in italics) reminding both of imminent death and further resurrection (that’s “Finnegan’s wake” as "a meeting after a funeral of the hero) and last seconds of a dream on the brink of waking up (that’s “Finnegans wake” as "heroes wake up"). And she also mentions Joyce in the liner notes thus underlining this parallel.
So this album represents some kind of closure for Joanna in this way as well, and where she goes next is really, truly impossible to predict.
That’s only some initial thoughts on the topic and I really hope that people with a better knowledge of Joyce’s work than me will be able to find something new and as of yet undiscovered in comparing the works of these two great artists.