PROJECT: LearnLanguages
AUTHOR: William Raiford (bill.mybiz@gmail.com)


VISION: I want the absolute best software to learn languages quickly, efficiently, and organically to maximize enjoyment and retention, and minimize time investment.  I also want a specific goal, currently 2000 words in each of 5 languages, and I want to see just _one_ real number metric to see my progress towards that goal in any particular language.  I want to be able to write in a bunch of text, have the app persist that text, and allow quick user actions to create translations in multiple languages to grow just a database of text and text relationships.  Eventually, I want to include item scheduling using a neural net to maximize retention and minimize repetitions necessary for learning.

SUMMARY: To start with, the algorithmic problem becomes three-fold:

1. Input text data ("Phrase"), associated with a Language.
2. Create text relationships (translations) among those text data.
3. Execute metrics on user knowledge of those text relationships, or IOW test for knowledge to assess progress.

NOTES ON BUILDING/RUNNING:
-Be sure to manually set the IISHost project as the StartUp Project.
-To switch between the Mock and Ef data providers: IISHost project -> Web.config -> <appsettings> -> key="DalManagerType".
-To switch between Silverlight Unit Testing UI and the actual Silverlight app UI: IISHost project -> Project Properties -> Web tab -> Specific Page(.TestsTestPage is the UnitTesting page right now - bad naming sorry).
-I usually work in a feature branch and then merge it back into master, so for most up-to-date code, check out the [feature] branch.

JOURNAL:
01/03/12
I've fleshed out PhraseEdit and LanguageEdit somewhat, but I'm still working on the asynchronization of my WCF calls.  Basic WCF service runs using BasicHttpBinding (because of SL requirements, it can't use WsHttpBinding).  

01/04/12
Restructured architecture to better reflect my increasing knowledge of setting up the data portal properly.  Instead of trying to host my own WCF service, I have created a IIS web page that exposes the Csla data portal service, so I can program a more proper Csla-way.

01/10/12
I've gotten tests passing for basic plumbing for the most basic structure.  I have a PhraseEdit and a LanguageEdit, and I have plumbing to CRUD those objects using EF.  Now, I am going to be focusing on authentication.

01/19/12
I've integrated authentication into the main LearnLanguages project.  This includes the software architecture with PhraseEdit and LanguageEdit, as well as the Ef database structure/model, including FK associations. I also have some basic pub-sub navigation in place as well, handled by Caliburn event aggregator.  I have the Ef database being deleted and seeded, controlled by a setting in EfResources.  Currently all tests passing for basic CRUD functionality in LanguageTests and PhraseTests, and the SecurityTests are passing as well.

01/24/12
Basic functionality is working now.  I can login, add phrases, view phrases, edit phrases, delete phrases.  None of it is super evolved, ie you can't do everything very fast, and it is not efficient, but it is usable with my vision of how it is intended to be used.  I won't be adding a bunch of little words, I will be adding long texts, ie groups of words.  Some may be song lyrics, or poems, or whatever.  Then, it will be part of the process of "learning" these phrases that will break these phrases down into different atomic units.  

01/29/12
I've integrated TranslationEdit functionality into both Mock and Ef Data Providers now.  A translation is a collection of phrases, requiring at minimum two phrases.  In the UI, I can now add translations, view translations, edit translations, but not delete them (the functionality exists but just not in UI yet).  If deleting a _phrase_ causes a translation to contain only one or zero phrases, however, then the translation _is_ automatically deleted cascadingly.  I cannot add existing phrases to a translation yet.  Overall, making progress with basic functionality & plumbing, and almost to the point of the cooler stuff with the learning process.  Nothing is optimized for speed, all for functionality, so that is also probably on the horizon once I'm working with actual larger data sets.  I also anticipate wanting to download a backup of data file.  All in all going well.

2/16/12
Still working through learning process. I have the process itself down,
but I'm trying to get the distributed architecture going.  It's turning
out to be somewhat building my version of a service bus.  I've though about
converting over to NServiceBus from the IEventAggregator in Caliburn,
but that is just too big of a jump for me to handle when I feel the
immediate usefulness of the program is so close.  As far as the coding goes,
at first, I was unknowingly trying to turn everything into pub-sub as far
as the learning process goes, but now I see that you can think of just one of 
the study pieces (the top level study partner) as the one who communicates 
with the viewmodel through pub-sub.  Pub-sub is just one method of getting
the various pieces working in harmony, and it takes a bit more overhead
than just writing a method or even a subclassed implementor, as each step
which would normally be a getter or a method call is now an entire message
which must be accounted for if it gets lost.  Speaking of which, I'm 
developing this as a fire-and-forget model, assuming loss, using an 
"expiration date" as my eventual timeout mechanism...working in the dairy
department 3 years make expiration date very well known to me LOL.  Still
the process is a long one, but I think this is the final major milestone
before I can start using this program for personal use. 

02/29/12
Well, almost have it freaking working.  It's been very slow going, trying to get it together all of the features I've had to forego for getting it done in a reasonable timeframe.  I almost have it usable for myself as far as persisting some kind of knowledge about phrases, and interpreting this persistence. It looks like for right now, it will be storing a timestamp, a string, and a double for the extent of "knowledge" or "belief". Anything can use this PhraseBeliefEdit class, even if they don't use it for phrases I suppose.  Though, I would like in future versions to have a more generic belief class/belief structure that will work more easily with multiple neural networks.  Had some interesting thoughts about those, in terms of aggregating functional neural networks.  Still working through that though, thinking of what precisely each step of "training" a neural network means in terms of an infinitely recursive neural network and a broader meta-neural network.  But I don't care much for expounding further until I can focus on just that aspect of the program, other than to say that training is a resource allocation investment strategy; a resource for which every generation of neural networks (in something similar to the rtNeat algorithm) is competing.  This may be our "fundamental" unit of currency, or at least one implementation of a fundamental resource currency.  Other than that, it's just plodding ahead with this simpler stuff, though the learning process itself is shaping out to be pretty cool, and I believe will be massively effective in and of itself.  The idea is that each MLT (multi-line text) and line phrase will be analogous to the "item" concept in SuperMemo, in that the point is to be reviewing this whole compound entities.  That much was already known.  Now, when you forget something, it will essentially fracture the compound phrase into smaller pieces, each of which will become atomic until the larger MLT/line is "remembered"/"known".  There are other dynamics as well, such as somewhat ignoring the known/forgotten pattern of words when you are first learning them, and not jumping the gun in trying to establish these items' learning parameters.  This is akin to ignoring the spike in information when plugging in a sound jack, or any state transition, which indeed, learning a new item can be considered a state transition, and the spike is the resources in attaining equilibrium for those items' strength in the brain's environment.  That's enough for technical details anyway, I'm just glad to be "back" to coding.  I had to do a lot of white-boarding and thinking and other seemingly "non-work" things that didn't seem to be producing much.  I think I'm on the right track now though.

03/10/12
I've gotten the studier working a little bit better now.  It has some knowledge working, and some of the persistence mechanisms are in place, but that is not utilized yet.  I'm having a fun time figuring out how to troubleshoot such a distributed architecture, but I think part of my frustration is just my expectation for immediate results is unrealistic.  I have extremely high standards for what I want to accomplish, and that means it's a difficult program that will take time and I just gotta accept that.  It is very gratifying to see the aggregation going well, and I see the idea is a good one.  The odd translation mixups with smaller phrases provide good noise to the overall translation when the entire line comes together, where you go "Ahhh, that is what the sentence means together in context."  So essentially, it's looking freaking awesome, but it's very tedious to code what seems like seemingly simple logic.  The completely async world is not kind heh.  A little async here and there I've done, but this is a different beast.  At least I'm coding though, and not white-boarding as I understand what my near-term goals seem to be.

04/11/12
Well, I'm back to trying to get this going some more.  I now have the study partner studying both meaning and order, so you can just go to Edit -> Add a Song (in whatever language you want), and then choose Study -> A Song.  Choose your song and it will go through and auto-translate using Bing translator, study meaning and once you get more and more meaning, it will ask you more and more the line order.  Next line of business is to get it to use custom translations, where you can modify whatever the auto-translate tells you.  This way, obviously, you can correct the sometimes insane translations, and these translations should persist.  Once that is done, clean it up some more, then get the statistics started so we have an approximate number of words per language that we know.  With that, it is in its most ridiculously simplistic default form: usable but not very special.  But it's built with a lot of extensibility so that we can add cool things like neural networks, manual study partners, link in with youtube video urls, better looking views (won't be difficult heh).  Anyway, we'll see if we can just get the next step: the custom translations.

08/13/12
WOO HOOO I'M BACK!  I had my computer crash awhile back and I've been surfing the unemployed and f it all wave for a bit now but I'm back and gonna be gettin this sucker goin.  Remember, if you give me a coding job (bug checking, unit testing, whatever) I will work for MINIMUM WAGE!  It probably doesn't mean much but I did get an 800 in math SAT (1500 overall) and a 36 in Math ACT (33 overall) and 5th place in STATE math competition back in the day (about 15 years ago).  I didn't have the resources to finish college and have been bouncing around shit to shit for awhile now.  So with that ineffectual bs out of the way I'm getting my dev environment back up and going.  My first goal is to just get this sucker back to where it was.  Then I'm going to look into fixing up what I wanted to with the code a bit as I have mentioned in my previous post.  Utilizing custom translations, because the Bing translation API has switched over to Azure now and that adds complications.  So in the meantime, manual translations will be where it's at.  After I get that in place, I'll look at Azure again, or some other free service if I can find it.  I also will put in a "simple" knowledge metric to give a one number representation of how much of a language I "know".  I'm going to be devoting time to this and getting recertified as A+ to earn shit in that arena again, but that shit is slightly less shitty than the other shit jobs so that's how it's gonna be, assuming other shit doesn't shit up all over the place, which I'm sure it will.  W9rd up 2tha H0m3yZ.

09/04/12
Well, I've gotten quite a bit done. It is almost useable to the point of wanting to put it up online for a go. Hopefully I'll get to that in a couple weeks. Would be awesome if it were sooner, but I doubt it. Anyway, I can now almost use the study a song bit, and that is the main part of the app for right now. It has a mediocre busy notification system in place now, for displaying to the user when the app is chugging on one of the methods (especially useful for the ubiquitious async methods). I still need to add a few features, such as searching for something in the web (e.g. ctrl+click on a �word to translate it at http://www.microsofttranslator.com/?=xxxx), and an options page.  I also need to work on the admin functions, specifically the add/remove users bit. Also need at least a rudimentary capability to remove a song, if not edit it as well. Some other things as well. The bottom line is that I am currently using it to actually learn a song in French at the moment, and it is somewhat working.  Nothing fancy as far as a neural net goes, but I'm thinking that won't be available until I branch out a little bit more.  I need to get the fundamental aspects of the program down. I still am unsure about my default study partner, as all of the async just seems a little shaky. Though, it's not necessarily the async, just the recent experience of mine trying to troubleshoot a multi-threaded tight loop using a waitone handle. The async aspects of SL4 just don't seem very clean. So, before putting this up �����on an actual site, I am considering making a tag for the branch, and trying to migrate to SL5, so that at the very least, I can use the tasks functionality. Ah yes, that reminds me...I definitely need something that demonstrates my current level of knowledge in a language. This is absolutely crucial, as I still want the 2000 word vocabulary goal in each of the languages. But overall, it feels better to actually be able to run the app and actually learn from it. The StudyASong has a decent learning algorithm to start with. In the future, of course, I want it to be absolutely kickass awesome. And it will be.