Golden Time: Help Offers

[YoakeNoHikari]

Ah, I see.

I'm horrible with computers, so I don't think I could do something like that.

[rpapo]

Ah, I see.

I'm horrible with computers, so I don't think I could do something like that.

Though I went to the university to study Mechanical Engineering, and graduated with that degree, my job has always been computer programming. In fact, I am not the greatest of users of canned software for the simple reason that when I need the computer to do something, I usually write the program myself. I never used Microsoft Word before 2001 or so, though I've been using computers since 1977.

[hobogunner]

...what language is your Program in, Rpapo? would it be possible to get it? (I suppose language doesn't matter if it's an .exe. )

[rpapo]

...what language is your Program in, Rpapo? would it be possible to get it? (I suppose language doesn't matter if it's an .exe. )

I have no problem with sharing the program, though it is definitely a work in progress, with much yet to be done. It is written in C++, using Microsoft Visual Studio 2008. The program uses the EDICT file from WWWJDICT as it's dictionary, and takes a "hit it with a harder hammer" to parsing Japanese, inspecting each character of the Japanese text and finding the longest match against what is in the dictionary (or anything the program can conjugate from it). The result isn't too bad, though I have also included a copy of the JUMAN parser to provide a second opinion of how to parse the text. I have found that JUMAN does some things better, but has a smaller dictionary which I do not know how to extend. My parser is dumber, but has a huge dictionary to work with. Neither parser is perfect, and quite often they disagree with each other; sometimes both are wrong.

The main problem with the current version, though, is that it requires a huge amount of memory to run. It is an X64 application, and requires at least 6Gb of available memory on the system it is running on. Otherwise, memory swapping makes it run painfully slow.

I have a version that is more memory efficient in the works, but I've been spending most of my time simply trying to improve my personal understanding of Japanese rather than working on the program, so I haven't done much with it in a while. I use the current version to parse each page of TXT file as I get them transcribed from the books.

[hobogunner]

Well, I would certainly enjoy playing around with it. I wonder how hard it would work on 15 pages in word?

I usually get the romaji of the text in word through a thing online, write it down, then look up things I don't know and make the garbled mess into a sentence.

As to not make this any more off topic would it be hard to email or P.M. it to me? (don't know if the reference files are downloads or online.)

[Mystes]

Great, I've got 8GB.

Will translating improve Japanese, rpapo?

[hobogunner]

The way rpapo does it I'm sure, as he transcribes from a page into a file. Past that, I believe we do the same steps, and yes, it will. It just slows the learning curve because you're going at a translating pace as opposed to thnking on usage for that one word, but it still will certainly make you learn.

[rpapo]

Well, I would certainly enjoy playing around with it. I wonder how hard it would work on 15 pages in word?

The memory it consumes is entirely for the dictionary it constructs, and that does not depend on the size of the document you wish to parse.

As to not make this any more off topic would it be hard to email or P.M. it to me? (don't know if the reference files are downloads or online.)

I really ought to make a few simple changes to the program first. Right now the program is pretty much hard-coded as to which input file it expects, and as to which lines I want parse. I haven't bothered to give it much of a user interface, not even a real command-line interface, yet. But I can probably do something tomorrow. Right now it's time for me to sign off for the day, as I get up really early to go to work.

Here is an example of the program's output, from where I am at in Golden Time:

Code: Select all

Line 1622:
Original: ――ここにいるし、柳澤の書いたビックリマークが板書を書き写した部分にまで入り込んでいるのにキレそうになって、消しゴムでガシガシゴシゴシ消していってもいる。
   JUMAN: ―― ここ に いる し、 柳澤 の 書いた ビックリ マーク が 板書 を 書き写した 部分 に まで 入り込んで-いる のに キレ そう に なって、 消し ゴム で ガシガシゴシゴシ 消して-いって も-いる。
    Kana: ―― ここ に いる し、 やなぎさわ の かいた びっくり まーく が ばんしょ を かきうつした ぶぶん に まで はいりこんで-いる のに きれ そう に なって、 けし ごむ で ガシガシゴシゴシ けして-いって も-いる。
  Romaji: ―― koko ni iru shi, yanagisawa no kaita bikkuri maaku ga bansho o kakiutsushita bubun ni made hairikonde-iru noni kire sou ni natte, keshi gomu de gashigashigoshigoshi keshite-itte mo-iru.
    Definitions:
      いる = (v1,vi) (1) (uk) to be (of animate objects)/to exist/(2) to stay/(v1,aux-v) (3) (after the -te form of a verb) verb indicating continuing action or state (i.e. to be ..ing, to have been ..ing)/(P)/
      いる = (v1,vt) to cast/to mint/to coin/(P)/
      いる = (v1,vt) to shoot/(P)/
      いる = (v5r,vi) to get in/to go in/to come in/to flow into/to set/to set in/(P)/
      いる = (v5r,vi) to need/(P)/
      いる = (oK) (v1,vt) to cast/to mint/to coin/
      いる = (v5r) (poet) to set beyond the Western horizon (i.e. the sun)/
      いる = (v5r,vt) to parch/to fry/to fire/to broil/to roast/to boil down (in oil)/
      柳澤 (やなぎさわ) = (name) Yanagisawa
      書いた (かいた) = [Plain Positive Past] (v5k,vt) (1) to write/(2) to draw/(P)/
      ビックリ = (vs,adv) (1) (uk) (on-mim) to be surprised/to be amazed/to be frightened/to be astonished/(adj-f) (2) (uk) (on-mim) surprise (e.g. surprise party)/
      マーク = (n,vs) (1) mark/(2) (abbr) markup/(3) (comp) Mach (OS kernel)/(P)/
      板書 (ばんしょ) = (n,vs) writing on a blackboard/
      書き写した (かきうつした) = [Plain Positive Past] (v5s,vt) to transcribe/
      部分 (ぶぶん) = (n) portion/section/part/(P)/
      入り込んで (はいりこんで) = [Plain Conjunctive] (v5m,vi) (1) to go into/to come into/to penetrate/to get in/to step in (a house)/(2) to become complicated/(P)/
      入り込んで (いりこんで) = [Plain Conjunctive] (v5m,vi) (1) to go into/to come into/to penetrate/to get in/to step in (a house)/(2) to become complicated/
      キレ = [Stem] (v1,vi) (col) to get angry/to snap/to blow one's top/to lose one's temper/to flip/
      なって = [Plain Conjunctive] (v5r,vi) (1) (uk) to become/to get/to grow/to be/to reach/to attain/(2) to result in/to prove to be/(3) to consist of/to be composed of/(4) to succeed/to be complete/(5) to change into/to be exchanged for/(6) to play a role/(7) to be promoted (shogi)/(v5r) (8) (hon) to do .../(P)/
      なって = [Plain Conjunctive] (v5r,vi) (uk) to bear fruit/(P)/
      なって = [Plain Conjunctive] (v5r,vi) to sound/to ring/to resound/to echo/to roar/to rumble/(P)/
      なって = [Plain Conjunctive] (suf,v5u) (after a noun, etc.) to do .../
      なって = [Plain Conjunctive] (v5r,vi) (1) (uk) to become/to get/to grow/to be/to reach/to attain/(2) to result in/to prove to be/(3) to consist of/to be composed of/(4) to succeed/to be complete/(5) to change into/to be exchanged for/(6) to play a role/(7) to be promoted (shogi)/(v5r) (8) (hon) to do .../
      なって = [Plain Conjunctive] (v5u) to twist/to twine/
      消し (けし) = [Stem] (v5s,vt) (1) to erase/to delete/to cross out/(2) to turn off power/(3) to extinguish/to put out/(4) (sl) to bump off/(P)/
      消し (しょうし) = [Stem] (vs-s,vi) (1) to disappear/(vs-s,vt) (2) to cause to disappear/to get rid of/(3) (of time) to pass/to while away/
      ゴム = (ateji) (n,adj-no) (1) (uk) gum (dut: gom)/rubber/(2) (abbr) eraser/(3) (sl) condom/
      消して (けして) = [Plain Conjunctive] (v5s,vt) (1) to erase/to delete/to cross out/(2) to turn off power/(3) to extinguish/to put out/(4) (sl) to bump off/(P)/
      消して (しょうして) = [Plain Conjunctive] (vs-s,vi) (1) to disappear/(vs-s,vt) (2) to cause to disappear/to get rid of/(3) (of time) to pass/to while away/
    Kana: ――ここに いる し、 やなぎさわ の かいた ビックリマーク が ばんしょ を かきうつした ぶぶん に まで はいりこんでいる のに キレ そうに なって、 けしゴム で ガシガシ ゴシゴシ けして いって も いる。 
    Roma: ――kokoni iru shi, yanagisawa no kaita bikkurimaaku ga bansho wo kakiutsushita bubun ni made hairikondeiru noni kire souni natte, keshigomu de gashigashi goshigoshi keshite itte mo iru. 
    Definitions:
      ここに = (adv) individually/separately/
      いる = (v1,vi) (1) (uk) to be (of animate objects)/to exist/(2) to stay/(v1,aux-v) (3) (after the -te form of a verb) verb indicating continuing action or state (i.e. to be ..ing, to have been ..ing)/(P)/
      いる = (v1,vt) to cast/to mint/to coin/(P)/
      いる = (v1,vt) to shoot/(P)/
      いる = (v5r,vi) to get in/to go in/to come in/to flow into/to set/to set in/(P)/
      いる = (v5r,vi) to need/(P)/
      いる = (oK) (v1,vt) to cast/to mint/to coin/
      いる = (v5r) (poet) to set beyond the Western horizon (i.e. the sun)/
      いる = (v5r,vt) to parch/to fry/to fire/to broil/to roast/to boil down (in oil)/
      し = (n) poem/verse of poetry/(P)/
      し = (n,n-suf) (1) death/decease/(2) (arch) death penalty (by strangulation or decapitation)/(P)/
      し = (n,n-suf) city/(P)/
      し = (n-suf,ctr) (abbr) magazine/(P)/
      し = (num) four/(P)/
      し = (prt,conj) (at the end of a phrase) notes one (of several) reasons/(P)/
      し = (ctr) counter for long, thin things (i.e. swords)/
      し = (n) (1) (arch) that/(2) you/(3) oneself/themself/
      し = (n) (1) child (esp. a boy)/(2) viscount/(3) (hon) master (founder of a school of thought, esp. Confucius)/(4) philosophy (branch of Chinese literature)/non-Confucian Hundred Schools of Thought writings/(5) (arch) you (of one's equals)/(n-suf) (6) -er (i.e. a man who spends all his time doing...)/
      し = (n) (1) corpse/
      し = (n) (1) messenger/(2) (abbr) police and judicial chief (Heian and Kamakura periods)/(3) (Buddh) klesha (polluting thoughts such as greed, hatred and delusion, which result in suffering)/
      し = (n) (1) printing block/
      し = (n) (1) teacher/master/one's mentor/(n,suf) (2) religious leader/(suf) (3) specialist/(n) (4) (arch) five-battalion brigade comprising 2500 men (Zhou-dynasty Chinese army)/
      し = (n) (1) thread/(2) one ten-thousandth/0.001 percent (one-ten-thousandth of a wari)/
      し = (n) (1) words/writing/lyrics/(2) ci (form of Chinese poetry)/(3) independent word/
      し = (n) (arch) office (government department beneath a bureau under the ritsuryo system)/
      し = (n) (obsc) 10**24/septillion (American)/quadrillion (British)/
      し = (n) Chinese "Turtle Beak" constellation (one of the 28 mansions)/
      し = (n) calling card/
      し = (n) official/civil service/
      し = (n) zhi (large bowl-shaped ancient Chinese cup with two handles)/
      し = (num) four (used in legal documents)/
      し = (num) four/
      し = (ok) (n) (uk) Japanese dock (plant) (Rumex japonicus)/
      し = (pref) to .../
      し = (pref,suf) visual/vision/
      し = (suf) (1) (hon) Mr/(n) (2) clan/(3) (hon) he/him/(ctr) (4) (hon) counter for people/
      し = (suf) (hon) honorific suffix used after the name of a woman of equal or higher status/
      し = (suf) history (e.g. of some country)/
      柳澤 (やなぎさわ) = (name) Yanagisawa
      の = (prt) (1) (uk) indicates possessive/(2) verb and adjective nominalizer (nominaliser)/(3) substituting for "ga" in subordinate phrases/(4) indicates a confident conclusion/(5) emotional emphasis (sentence end) (fem)/(6) indicates question (sentence end)/(P)/
      の = (n) (1) (arch) shaft (of an arrow)/(2) (arch) arrow bamboo (Pseudosasa japonica)/
      の = (n) (1) plain/field/(2) hidden (structural) member/(n-pref) (3) wild/
      の = (prt) (arch) possessive (used on tombs, etc.)/
      の = (prt) indicates possessive among other uses (for full details and examples see the main entry (linked))/
      書いた (かいた) = [Plain Positive Past] (v5k,vt) (1) to write/(2) to draw/(P)/
      ビックリマーク = (n) exclamation mark/exclamation point/
      が = (prt) (1) indicates sentence subject (occasionally object)/(2) indicates possessive (esp. in literary expressions)/(prt,conj) (3) but/however/still/and/(P)/
      が = (n) (1) (Buddh) obstinacy/(2) atman/the self/the ego/
      が = (n) congratulation/
      が = (n) moth/
      が = (n) picture/drawing/painting/sketch/
      が = (n,adj-na) (1) (arch) elegance/grace/(n) (2) festal song (genre of the Shi Jing)/
      板書 (ばんしょ) = (n,vs) writing on a blackboard/
      を = (prt) (1) indicates direct object of action/(2) indicates subject of causative expression/(3) indicates an area traversed/(4) indicates time (period) over which action takes place/(5) indicates point of departure or separation of action/(6) indicates object of desire, like, hate, etc./(P)/
      書き写した (かきうつした) = [Plain Positive Past] (v5s,vt) to transcribe/
      部分 (ぶぶん) = (n) portion/section/part/(P)/
      に = (prt) indicates such things as location of person or thing, location of short-term action, etc./(P)/
      に = [Stem] (v1,vi) (1) to resemble/to look like/to take after/(2) to be similar (in status, condition, etc.)/to be close/to be alike/to be like/(P)/
      に = [Stem] (v1,vt) to boil/to simmer/to stew/to seethe/(P)/
      に = (n) (arch) soil (esp. reddish soil)/
      に = (n,n-suf) (abbr) bhikkhuni (fully ordained Buddhist nun)/
      に = (num) two (used in legal documents)/
      に = (num) two/
      に = (suf) takes after (his mother)/
      まで = (prt) (uk) until/till doing/as far as/(P)/
      入り込んでいる (はいりこんでいる) = [Plain Progressive] (v5m,vi) (1) to go into/to come into/to penetrate/to get in/to step in (a house)/(2) to become complicated/(P)/
      入り込んでいる (いりこんでいる) = [Plain Progressive] (v5m,vi) (1) to go into/to come into/to penetrate/to get in/to step in (a house)/(2) to become complicated/
      のに = (prt) (1) although/when/and yet/despite this/in spite of/even though/but even so/but even then/however/nevertheless/for all that/notwithstanding that/(2) while/(3) if only/I wish/(4) I tell you/you should do/(5) in order to/(P)/
      キレ = [Stem] (v1,vi) (col) to get angry/to snap/to blow one's top/to lose one's temper/to flip/
      そうに = [Adverb] (adv) (1) so/really/seeming/(adj-na,suf) (2) (after masu stem and adj-stem) appearing that/seeming that/looking like/having the appearance of/(P)/
      そうに = (n) monks and nuns/
      そうに = [Adverb] (n,adj-na) (1) vibrancy/strength/bravery/manliness/(2) (esp. of men) one's prime (approx. age 30)/
      なって = [Plain Conjunctive] (v5r,vi) (1) (uk) to become/to get/to grow/to be/to reach/to attain/(2) to result in/to prove to be/(3) to consist of/to be composed of/(4) to succeed/to be complete/(5) to change into/to be exchanged for/(6) to play a role/(7) to be promoted (shogi)/(v5r) (8) (hon) to do .../(P)/
      なって = [Plain Conjunctive] (v5r,vi) (uk) to bear fruit/(P)/
      なって = [Plain Conjunctive] (v5r,vi) to sound/to ring/to resound/to echo/to roar/to rumble/(P)/
      なって = [Plain Conjunctive] (suf,v5u) (after a noun, etc.) to do .../
      なって = [Plain Conjunctive] (v5r,vi) (1) (uk) to become/to get/to grow/to be/to reach/to attain/(2) to result in/to prove to be/(3) to consist of/to be composed of/(4) to succeed/to be complete/(5) to change into/to be exchanged for/(6) to play a role/(7) to be promoted (shogi)/(v5r) (8) (hon) to do .../
      なって = [Plain Conjunctive] (v5u) to twist/to twine/
      消しゴム (けしゴム) = (n) eraser/India rubber/(P)/
      で = (prt) (1) indicates location of action/at/in/(2) indicates time of action/(3) indicates means of action/cause of effect/by/(conj) (4) and then/so/(aux) (5) indicates continuing action/(P)/
      で = [Stem] (v1,vi) (1) to go out/to exit/to leave/(2) to appear/to come forth/to be published/(3) to answer (the phone or door)/(P)/
      ガシガシ = (adv,adv-to) (on-mim) boisterously/roughly/briskly/(P)/
      ゴシゴシ = (n,vs,adv) (on-mim) scrubbing/rubbing briskly/rasping/
      消して (けして) = [Plain Conjunctive] (v5s,vt) (1) to erase/to delete/to cross out/(2) to turn off power/(3) to extinguish/to put out/(4) (sl) to bump off/(P)/
      消して (しょうして) = [Plain Conjunctive] (vs-s,vi) (1) to disappear/(vs-s,vt) (2) to cause to disappear/to get rid of/(3) (of time) to pass/to while away/
      いって = (n) move (in game)/method/single-handed/monopoly/(P)/
      いって = [Plain Conjunctive] (v5r,vi) to get in/to go in/to come in/to flow into/to set/to set in/(P)/
      いって = [Plain Conjunctive] (v5r,vi) to need/(P)/
      いって = [Plain Conjunctive] (v5u) (1) to say/(2) to call (i.e. to give a name)/(P)/
      いって = [Plain Conjunctive] (v5r) (poet) to set beyond the Western horizon (i.e. the sun)/
      いって = [Plain Conjunctive] (v5r,vt) to parch/to fry/to fire/to broil/to roast/to boil down (in oil)/
      いって = [Plain Conjunctive] (v5u) (1) to say/(2) to call (i.e. to give a name)/
      いって = [Plain Conjunctive] (v5u,vt) to do up (hair)/to braid/to fasten/to fix/
      も = (prt) (1) also/too/words of similar weight/(2) about (emphasizing an upper limit)/as much as/even/(adv) (3) more/further/other/again/(P)/
      も = (n) ancient skirt/
      も = (ok) (n) (1) face/(2) surface/

[rpapo]

The way rpapo does it I'm sure, as he transcribes from a page into a file. Past that, I believe we do the same steps, and yes, it will. It just slows the learning curve because you're going at a translating pace as opposed to thnking on usage for that one word, but it still will certainly make you learn.

This project has absolutely helped my understanding of Japanese, especially written Japanese. For spoken I have to rely on anime, movies and language tapes.

To put things in perspective, when I started helping on B-T, on Toradora Spinoff 2, it took me two weeks to do the Author's Notes, which was less than two pages in all. Now, it takes me about 45 minutes to transcribe a page, and one to two hours to translate that page . . . unless I get hung up on something. That still happens every 4-5 pages or so. Reviewing the "Golden Time" forum topic will show that quite clearly.

[hobogunner]

Wow, that is nearly what I need to translate, that should help my translation speed too.

You don't have to spend time making a UI if you don't want too, I have enough of an undertanding to know how to do it manually.

6GB for the dictionaries.....geez, my computer is built for it, but when I gave iTunes full access to sync all my photos, it used 100% of my quad-core and overheated. does it take so much of your processor that I should limit it to 1 or 2?

[Doraneko]

The 2D-kun spin-offs of Golden Time also makes a viable choice for shorter translation projects. In fact they can be enjoyed without knowledge of the main story. The three shorts make a coherent medium-length comedy on how an otaku finds his love (?).

Off topic: Talking about Toradora translations, some time ago I came across this.

[jonathanasdf]

also off topic: was just wondering how similar that program is to Translation Aggregator, which is what I'm currently using..

[YoakeNoHikari]

Off topic: Talking about Toradora translations, some time ago I came across this.

I laughed so hard.

[rpapo]

Wow, that is nearly what I need to translate, that should help my translation speed too.

That was the idea. I wrote it to automate as fully as possible the text parsing and dictionary lookup phase of translation, since a perfectly good electronic dictionary was available for free (WWWJDICT). Better yet, when I find worthwhile additions to the dictionary, I can submit them to their web site. They don't usually take my suggestions verbatim, but rather correct them, which is fine with me. They're the gurus, after all. I've submitted probably 35-40 words/phrases to the dictionary over time, and they've accepted most of them.

Another thing I can do with my own parser is extend the dictionary with colloquial phrases and personal and location names.

You don't have to spend time making a UI if you don't want too, I have enough of an undertanding to know how to do it manually.

Well, without a UI you must have Visual Studio 2008 to modify and recompile the code each time you want to change the running parameters (the input file, the output file, the starting line and ending line for the run).

6GB for the dictionaries.....geez, my computer is built for it, but when I gave iTunes full access to sync all my photos, it used 100% of my quad-core and overheated. does it take so much of your processor that I should limit it to 1 or 2?

I haven't had a computer meltdown yet...

The 2D-kun spin-offs of Golden Time also makes a viable choice for shorter translation projects. In fact they can be enjoyed without knowledge of the main story. The three shorts make a coherent medium-length comedy on how an otaku finds his love (?).

Agreed, 100%.

also off topic: was just wondering how similar that program is to Translation Aggregator, which is what I'm currently using..

I'm not familiar with it. Like I said before, I tend to just write what I need rather than search the internet for something that probably doesn't quite work the way I want. In part, that has do with the fact that the internet itself didn't become a factor in my work until after I'd already been programming for 15 years or so.

FWIW, on another (albeit related) topic, I just received a request for permission (as if they needed it) from some Spanish speakers who want to move ahead translating Golden Time from English to Spanish. They wanted to know if I could verify their translations afterwards to make sure they hadn't drifted too much from the original meaning. Looks like this project may soon be getting a second language. The current translation of the Prologue is mine, but I've been trying to spend my time working more on the Japanese to English part. I'm already fluent in Spanish...

[rpapo]

Another factoid regarding my parser program: the dictionary itself, as it comes from WWWJDICT, is large but not monstrous. It has approximately 150K entries in it. What makes my program's dictionary so huge is that my program conjugates everything it can, verbs and adjectives, in many ways, and adds each conjugation back into the dictionary for quick lookup. The result is a dictionary with about 2.5 million entries (rather large). My revised parser (as yet incomplete) would do conjugations on the fly, so it will probably be able to run on 32-bit machines without problems while covering yet more of the possible conjugations... once I get back to working on it.

Colloquial speech really makes my parser stumble, unless I add such word forms into the dictionary directly.