New Baka-Tsuki Android App: Development

[Taper]

For some reason the app is having issues with update service. Whenever it checks for chapter updates (whether it's done automatically or manually), it starts fine and checks normally, but after a while it just... hangs. No matter what I do, the updater won't resume, stop, or start so I give up on it. Then sometime MUCH LATER it finally finishes. This happens on both my android devices (HTC G2 and Galaxy Tab), so it's not device-side issue. It's not issue of bandwidth either, as all other services using bandwidth are accessing data normally.

Actually, now that I think about it, it almost always hangs at exact same point during the update process. It stops at checking chapter 90 of 91 of some novel, but I don't know which novel it is.

[animeout]

Hey, needed some advice from you

I am creating a php-base unofficial baka-tsuki API, which is basically parsing using HTMLDOM (like done in the android app) and making it available via a REST interface.

I am a bit stuck on getting the chapters and their links part, I was using the following DOM and it works with 70-80% of the LNs but doesnt work with some

Code: Select all

html body div div div dl dd ul

Basically, the problem is that on some LN listings, their chapters (name and links) do not have the dl dd DOM and on some LN listings there are additional links at top with dl dd DOM tags, like different language page links, which breaks the parser

Can you give me some tips or let me know how are you parsing the chapters in a safe way ?

[Taper]

Actually, now that I think about it, it almost always hangs at exact same point during the update process. It stops at checking chapter 90 of 91 of some novel, but I don't know which novel it is.

After keeping eye on the updates, it stops at Hagure Yuusha no Aesthetica "Downloading chapter info 91 of 92..."

Edit: An update, once I removed Hagure Yuusha no Aesthetica, the updater continued normally until it hanged once more again at Kill no More. "Downloading chapter info 0 of 107..." Removing that allowed it to update until stalling at Tate no Yuusha no Nariagari, "Checking: Chapter 86 - 愚痴 - Murmurs - PREVIEW - ===>LINK". Actually, it looks more like it froze? I'll check again later when I have time once LN is removed.

Also, I noticed that for some reason HEAVY OBJECT is listed as first entry for LNs that start with letter H, despite there being several LNs starting with "Ha-".

[JRave]

Actually, now that I think about it, it almost always hangs at exact same point during the update process. It stops at checking chapter 90 of 91 of some novel, but I don't know which novel it is.

After keeping eye on the updates, it stops at Hagure Yuusha no Aesthetica "Downloading chapter info 91 of 92..."

Edit: An update, once I removed Hagure Yuusha no Aesthetica, the updater continued normally until it hanged once more again at Kill no More. "Downloading chapter info 0 of 107..." Removing that allowed it to update until stalling at Tate no Yuusha no Nariagari, "Checking: Chapter 86 - 愚痴 - Murmurs - PREVIEW - ===>LINK". Actually, it looks more like it froze? I'll check again later when I have time once LN is removed.

Also, I noticed that for some reason HEAVY OBJECT is listed as first entry for LNs that start with letter H, despite there being several LNs starting with "Ha-".

Glad to see I am not the only one having issues with updating. I keep hanging at "Absolute Duo Downloading chapter info 0 of 107" and now Ark 0 of 109. If I tap Update Now again, it just hangs at Update Status: Running. I've been having issues with the app for a few updates now, but this has basically killed it for me. At this point it is just easier to read the projects in my browser vs using the app.

[Lionspork]

There seems to be some issue with downloading content when connecting using HTTPS. Text downloads fine but thumbnails seem to be broken, until HTTPS is unchecked. This issue is present both with and without using application keystore, and occurs both when "downloading chapter" and when viewing chapter.

[oogav]

I'm not sure if this is related to people having their updates hang, but I woke up this morning to a message of my phone having gone over my monthly data limit. That being extremely strange, I checked out the data usage and saw that BakaReader used almost 2GB of data overnight:

Spoiler! :

I've been using BakaReader since the old app and haven't encountered this issue. Just something I'd give a heads-up about.

[Taper]

I'll confirm that the updater worked normally with those watched LNs removed. However I once more again ran into hanging issue with one of LNs (that I didn't have chance to check) until it finally finished some time later.

After oogav's post above, I got curious and checked the bandwidth usage by the app and was surprised to see that it had used more than 30GB of data in just past two weeks- more than 10% of my monthly data allowance. Just today alone the app used 3GB. I don't even have large image download option enabled. While I do have about 100 LNs on my watch list, I don't see how that could pull so much bandwidth just to check whether content has been changed (if I'm not mistaken, the app checks for differences in timestamps between the downloaded content and the BT server).

I've now disabled the app's updater functionality, as well as my tablet's WiFi whenever I'm not using it just in case.

[Nandaka]

Yo, first time posting here. i found a Problem in the app, when i access a external novel, like Hyoketsu no Kyoukai, i could read it the first time,when I reaccess it, all I get was a blank page. I saved the external page as a archive in the app so i can access it offline but i also get a blank page then.

Most likely because of redirect issue, the link redirect to the mobile page using blank redirection. May need to check this.

Updating issues...

I'm not sure if this is because of Baka-Tsuki upgrade issue, can someone give me the android log file?

There seems to be some issue with downloading content when connecting using HTTPS. Text downloads fine but thumbnails seem to be broken, until HTTPS is unchecked. This issue is present both with and without using application keystore, and occurs both when "downloading chapter" and when viewing chapter.

Yeah, I think the certificate got an issue. My PC Chrome browser shows Invalid Certificate Warning.
See http://i.imgur.com/oVrXMuU.png

I'm not sure if this is related to people having their updates hang, but I woke up this morning to a message of my phone having gone over my monthly data limit. That being extremely strange, I checked out the data usage and saw that BakaReader used almost 2GB of data overnight:

Spoiler! :

I've been using BakaReader since the old app and haven't encountered this issue. Just something I'd give a heads-up about.

I'll confirm that the updater worked normally with those watched LNs removed. However I once more again ran into hanging issue with one of LNs (that I didn't have chance to check) until it finally finished some time later.

After oogav's post above, I got curious and checked the bandwidth usage by the app and was surprised to see that it had used more than 30GB of data in just past two weeks- more than 10% of my monthly data allowance. Just today alone the app used 3GB. I don't even have large image download option enabled. While I do have about 100 LNs on my watch list, I don't see how that could pull so much bandwidth just to check whether content has been changed (if I'm not mistaken, the app checks for differences in timestamps between the downloaded content and the BT server).

I've now disabled the app's updater functionality, as well as my tablet's WiFi whenever I'm not using it just in case.

How often do you set the update frequency? Do you include the external chapter when downloading? Text contents should not hit that much...

I'll add a feature to only update using wifi on the next release.

Hey, needed some advice from you

I am creating a php-base unofficial baka-tsuki API, which is basically parsing using HTMLDOM (like done in the android app) and making it available via a REST interface.

I am a bit stuck on getting the chapters and their links part, I was using the following DOM and it works with 70-80% of the LNs but doesnt work with some

Code: Select all

html body div div div dl dd ul

Basically, the problem is that on some LN listings, their chapters (name and links) do not have the dl dd DOM and on some LN listings there are additional links at top with dl dd DOM tags, like different language page links, which breaks the parser

Can you give me some tips or let me know how are you parsing the chapters in a safe way ?

No safe way, you need to include handle those scenario OR ask the translator to change the formatting. If the page is not locked, you may need to change it by yourself...

If there is a way to tag the element with something (hint: try to use class attribute, e.g.: class='synopsis', class='book_list', class='chapters')



Also, do you guys hit a lot of HTTP ERROR 400 recently?

[Bleboo]

Hello Guys,

recently I have the problem that the novel illustrations in the chapters dont show up properly, even if i download the novel completely. I don't know how to fix this and re-downloaded the app several times to try new options in the app, but nothing works.

I sent you a screenshot of how it looks on my device (HTC One M8). Maybe you know a solution and can help me.

http://imgur.com/GoVHXw2

Thanks.

[Lionspork]

There seems to be some issue with downloading content when connecting using HTTPS. Text downloads fine but thumbnails seem to be broken, until HTTPS is unchecked. This issue is present both with and without using application keystore, and occurs both when "downloading chapter" and when viewing chapter.

Yeah, I think the certificate got an issue. My PC Chrome browser shows Invalid Certificate Warning.
See http://i.imgur.com/oVrXMuU.png

The annoying thing is that it works without an issue on my tablet (Galaxy Tab S 8.4 running Kitkat) but refuses to work on my phone (Galaxy S5 running Lollipop) and problem has persisted across factory resets, and I think it was also a problem on my previous phone (Galaxy S3 Mini running 4.1 Jellybean).
There isn't a problem downloading full images either, only thumbnails.
Would it help to have an android log or something, only thing is I don't know how to dump one.

Also, do you guys hit a lot of HTTP ERROR 400 recently?

Yes, have to use HTTPS to make it work (leaving me without thumbnails)

[Lionspork]

Hello Guys,

recently I have the problem that the novel illustrations in the chapters dont show up properly, even if i download the novel completely. I don't know how to fix this and re-downloaded the app several times to try new options in the app, but nothing works.

I sent you a screenshot of how it looks on my device (HTC One M8). Maybe you know a solution and can help me.

http://imgur.com/GoVHXw2

Thanks.

Hi Bleboo are you connecting with HTTPS? Do the full illustrations still show up properly if you tap on the broken thumbnail? Because that sounds like the problem I'm having.

[Lionspork]

Sorry for the repeated posts but from what I seem to have determined the thumbnails are in fact downloading download properly through HTTPS (it shows up in the cache when I check in a file explorer) but the thumbnail fails to display inline if HTTPS is on. Turning HTTPS off makes the thumbnail show up again, even if there is no Internet connection.

[Lionspork]

Okay, I might have been a little wrong about that last one, switching off HTTPS doesn't always reliably make the thumbnails display properly (though switching it on does reliably break them). Although looking in /storage/sdcard0/Android/data/com.erakk.lnreader/files/project/images/thumb does show the 180px thumbnail JPGs themselves to be present in the correct folders.

[voker]

Hi, I wonder how can fix the [Bitmap Empty] problem. I download the novel completely, but it keep appear [Bitmap Empty] after I open the novel. And novel Illustrations in the chapters don't show up properly...

[Lionspork]

Hmm, it seems as though for a thumbnail to work there needs to have been, at least once, a working internet connection with HTTPS turned off (even if the entire volume has allegedly been downloaded) when viewing a particular thumbnail. If HTTPS is turned on or if there is no viable HTTP connection (like if using flight mode or if BT is throwing out Error 400) then the thumbnail is broken. Even though the thumbnail shows up in her cache.