Tuesday 7 February 2017

Chop the head off - making a headless Haven and Hearth client

Today I figured out I need to cut the head off Amber Haven and Hearth client. Main reason for it is that JOGL doesn't really like raspberry pi - there are lots of forum discussions about raspberry pi and JOGL, a couple suggested solutions, but unfortunately none of them really worked for me. So what we are going to try to do today is introduce a way to start Haven and Hearth client without visible window and lets hope it solves the problem.

Find the neck

First of all we need to identify places we need to modify in order to supress window creation. This should be a pretty simple process - we build our client on raspberry pi and run it:
This gives us two places in MainFrame.java where we crash, so lets move them under a condition, which would be a command line option, so we'll get something like:
        add(p);
        if (!Config.headless)
        	pack();
        setResizable(!Utils.getprefb("wndlock", false));
        p.requestFocus();
        seticon();
        if (!Config.headless)
        	setVisible(true);
        p.init();
        addWindowListener(new WindowAdapter() {

Obviously we need to add "headless" as a field in Config.java and modify Config.cmdline() to set that field.
public class Config {
...
	public static boolean headless = false;
...
    public static void cmdline(String[] args) {
        PosixArgs opt = PosixArgs.getopt(args, "hdPGp:U:r:A:u:C:l:");
...
                case 'l':
                	headless = true;
                	System.out.println("Running Headless - enjoy!");
                	break;
...
Ok, now we have an option lets compile and try to use it, we might want to modify run.bat at some point, but for now lets just run the client from command line manually.
java -Xms512m -Xmx1024m -jar hafen.jar -U http://game.havenandhearth.com/hres/ game.havenandhearth.com -l true
This unfortunately still doesn't give us a stable state:
So, we'll get this exception under condition as well, like this:
        public void run() {
            try {
                uglyjoglhack();
                if (state == null){
                	if(!Config.headless)
                          throw (new RuntimeException("State applier is still null after redraw"));
                	else
                	  System.out.println("State applier is still null after redraw");
                	
                }

Now we got our headless client running, but it is still pretty useless, since there is no we can interact with it.

I have no mouth and I have to scream

So, we need a way to tell client what we want from it without interacting with graphical interface, so lets create a console interface! Our console interface is going to be pretty simple - it will read a command from standard input and will pass it to root console:
package haven.automation;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;

import haven.HavenPanel;

public class ConsoleInterface implements Runnable {

	@Override
	public void run() {
		// TODO Auto-generated method stub
		BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
		String cmd = null;
		while(true){
			System.out.print("?> ");
			try {
				cmd = br.readLine();
			} catch (IOException e) {
				// TODO Auto-generated catch block
				e.printStackTrace();
			}
			System.out.println("< "+cmd);
			try {
				HavenPanel.lui.root.ui.cons.run(cmd);
			} catch (Exception e) {
				// TODO Auto-generated catch block
				e.printStackTrace();
			}
		}

	}

}
We need to remember to start our ConsoleInterface thread, so something like this:
...
        this.g = new ThreadGroup(HackThread.tg(), "Haven client");
        this.mt = new HackThread(this.g, this, "Haven main thread");
        this.ct = new Thread(new ConsoleInterface(),"Haven Console Interface");
        this.ct.start();
...
Now lets try it out - we will be using our console interface together with graphical interface for demonstration purposes.

Ways to go

This enables us with a number of interesting further developments - next thing we should do is enable our ConsoleInterface to receive command output, and then extend it with additional commands to interact with objects and subjects.


Updating JOGL to 2.3.2 in Amber - Haven and Hearth Client because we can

Today I was looking at Amber Haven and Hearth client and thing I noticed was that it was running some really old JOGL (library used to do graphics - OpenGL). So I thought - why would I want to run something from 2013 - most likely JOGL supplied with Amber is 2.1.2 released 1 November 2013.

Get your libraries

Amber has JOGL libraries predownloaded into "lib" directory and uses slightly different naming notation than the actual JOGL release.
So we need to update "gluegen-rt.jar" and "jogl.jar" in lib folder and then download native library jar's into "jogl-natives" folder.  Note that on the above screenshot we already have updated libraries, plus I downloaded armv6hf since I am playing with Haven and Hearth client on Raspberry Pi. Once we identified which libraries we need we can download them from https://jogamp.org/deployment/v2.3.2/

A couple of things worth mentioning - as stated above JOGL release uses a slightly different naming scheme, so "jogl.jar" in Amber client is "jogl-all.jar" in JOGL release, and "jogl-natives-X-Y.jar" on Amber side matches "jogl-all-natives-X-Y.jar" on JOGL release side.

There are several pitfalls that we need to avoid:
  1. All libraries should be from the same version - we can not use library of one release and native library from another that would cause a "Certificate mismatch" exception when trying to run the client
  2. Naming convention should be intact - build.xml expects particular jar names:

  3. jogl-jar and gluegen.jar can come double compressed, so you might need to extract jar from jar:

Fixing errors

Once we are done updating libraries and try to compile we are going to get multiple errors majority of which are going to be of two types:
    [javac] C:\workspace\amber\src\haven\HavenPanel.java:43: error: package javax.media.opengl does not exist
    [javac] import javax.media.opengl.*;
    [javac] ^
    [javac] C:\workspace\amber\src\haven\HavenPanel.java:71: error: cannot find symbol
    [javac]     private static GLCapabilities stdcaps() {
    [javac]                    ^
    [javac]   symbol:   class GLCapabilities
    [javac]   location: class HavenPanel
We are going to fix those that are about packages that do not exist. Reason for those errors is that in new libraries (not sure when the change happened, but who cares and why?) have "opengl" moved from "javax.media" package to "com.jogamp".

Once all packages are fixed there are two more things to be addressed. One of them are screenshots, it happens so, that com.jogamp.opengl.util.awt.Screenshot was deprecated in JOGL 2.x and is completely removed in 2.3.2. To address this we will have to modify "takescreenshot()" method defined within an anonymous GLEventListener object in initgl() method in HavenPanel.

            public void takescreenshot(GLAutoDrawable d) {
                try {
                    String curtimestamp = new SimpleDateFormat("yyyy-MM-dd_HH.mm.ss.SSS").format(new Date());
                    File outputfile = new File(String.format("screenshots/%s.png", curtimestamp));
                    outputfile.getParentFile().mkdirs();
                    AWTGLReadBufferUtil glReadBufferUtil=new AWTGLReadBufferUtil(d.getGLProfile(),false);
                    BufferedImage image=glReadBufferUtil.readPixelsToBufferedImage(d.getGL(),true);
                    ImageIO.write(image, "png", outputfile);                  
                    ui.root.findchild(GameUI.class).msg(String.format("Screenshot has been saved as \"%s\"", outputfile.getName()), Color.WHITE);
                } catch (Exception ex) {
                    System.out.println("Unable to take screenshot: " + ex.getMessage());
                }
            }

            public void display(GLAutoDrawable d) {
                if (HavenPanel.needtotakescreenshot) {
                    takescreenshot(d);
                    HavenPanel.needtotakescreenshot = false;
                }

As you can see, we no longer pass width and height into functional and we use AWTGLReadBufferUtil to get image data and then save it into the file via ImageIO.write().

This should leave us with two issues in GLProgram.create() and GLShadre.create() methods. Root cause of these issues is that gl.glCreateProgramObjectARB() and gl.glCreateShaderObjectARB() no longer return and Integer type, but Long type - so we'll cast result to Integer - this might not be perfect and might cause some issues going forward, but that seems to be the easiest fix, since changing type of id to Long is causing further incompatibilities.

Time to run

Ok, now we finally should be able to compile our code with new libraries and enjoy Haven and Hearth Client with JOGL 2.3.2

Thursday 2 February 2017

Setting things on Fire! ...in an automatic way

One of the benefits of having access to Haven and Hearth client source code is ability to automate certain routine tasks - to make them less routine or more efficient or both. So why don't we try automating something?

Task Definition

We have Fireplaces and Stockpiles - Fireplaces burn, Stockpiles store fuel (wood blocks) for our Fireplaces. We want to make sure that we have a couple of wood-blocks in inventory (no less than 3), we go to Fireplaces one by one, click-Open them, check what is the fuel level and if it is less than 50 - half, then we drop another wood block into the Fireplace. Relax for half an hour - repeat.

Implementation

We'll base our implementation on code that already exists (SteelRefueler.java) and will modify it for our use-case. So lets create a new class under "haven.automation" extending haven.Window and implementing haven.automation.GobSelectCallback
public class AutoFire extends Window implements GobSelectCallback {
 private static final Text.Foundry infof = new Text.Foundry(Text.sans, 10).aa(true);
 private static final Text.Foundry countf = new Text.Foundry(Text.sans.deriveFont(Font.BOLD), 12).aa(true);
 private List<Gob> fires = new ArrayList<>();
 private List<Gob> stockpiles = new ArrayList<>();
 private final Label lblf, lbls;
 public boolean terminate = false;
 private Button clearbtn, runbtn, stopbtn;
 private static final int TIMEOUT = 2000;
 private static final int HAND_DELAY = 8;
 private static final int SLEEP = 30 * 60 * 1000; // 30 min
 private Thread runner;
Lets have a look at what we do here:

  • infof - foundry for information labels
  • countf - foundry for counter labels
  • fires - array of graphic objects representing fireplaces
  • stockpiles - array of graphic objects representing stockpiles
  • lblf, lbls - labels to show number's of fireplaces and stockpiles configured
  • terminate - a terminate flag
  • clearbton, runbtn, stopbtn - objects for "Clear", "Run" and "Stop" buttons
  • TIMEOUT - two seconds of delay between actions that need to complete before the next action
  • HAND_DELAY - eight milliseconds of delay between attempts to use item in hand
  • SLEEP - delay between loops
  • runner - background thread running our automation
Now lets write a constructor for our class:
 public AutoFire() {
  super(new Coord(270, 180), "Auto Fire");

  final Label lbl = new Label("Alt + Click to select fireplaces and stockpiles.", infof);
  add(lbl, new Coord(30, 20));

  Label lblctxt = new Label("Fires Selected:", infof);
  add(lblctxt, new Coord(15, 60));
  lblf = new Label("0", countf, true);
  add(lblf, new Coord(110, 58));

  Label lblstxt = new Label("Stockpiles Selected:", infof);
  add(lblstxt, new Coord(135, 60));
  lbls = new Label("0", countf, true);
  add(lbls, new Coord(235, 58));

  clearbtn = new Button(140, "Clear Selection") {
   @Override
   public void click() {
    fires.clear();
    stockpiles.clear();
    lblf.settext(fires.size() + "");
    lbls.settext(stockpiles.size() + "");
   }
  };
  add(clearbtn, new Coord(65, 90));

  runbtn = new Button(140, "Run") {
   @Override
   public void click() {
    if (fires.size() == 0) {
     gameui().error("No fires selected.");
     return;
    } else if (stockpiles.size() == 0) {
     gameui().error("No stockpiles selected.");
     return;
    }

    this.hide();
    cbtn.hide();
    clearbtn.hide();
    stopbtn.show();
    terminate = false;

    runner = new Thread(new Runner(), "Auto Fire");
    runner.start();
   }
  };
  add(runbtn, new Coord(65, 140));

  stopbtn = new Button(140, "Stop") {
   @Override
   public void click() {
    terminate = true;
    // TODO: terminate PF
    this.hide();
    runbtn.show();
    clearbtn.show();
    cbtn.show();
   }
  };
  stopbtn.hide();
  add(stopbtn, new Coord(65, 140));
 }

So what is happening here? We create our window, call super() constructor setting it's coordinates and title to "Auto Fire". Then we create labels with respective content and place them around the window. Once that is done - we add our "Clear Selection", "Run" and "Stop" buttons placing them in the window and hiding the "Stop" button.

Each button has click() method overriden with functionality we want it to expose:

  • "Clear Selection" button would empty fires and stockpiles arrays and update text content of labels with appropriate numbers corresponding to new sizes of said array
  • "Run" button will trigger a check for fires and stockpiles arrays sizes and will present an error in case either of two is empty, then it will hide other buttons and show the "Stop" one and create a thread running automation logic
  • "Stop" button sets termnination attribute, hides itself and shows other buttons
As one can see constructor is pretty simple and revolves around setting up the window, defining labels and buttons and associating functionality with button clicks.


Now lets write the actual automation:
 private class Runner implements Runnable {

  @Override
  public void run() {
   GameUI gui = gameui();
   while (!terminate) {
    gui.syslog.append("Starting Loop!",Color.CYAN);
    floop: for (Gob f : fires) {
     // take fuel from stockpiles if we don't have enough in the
     // inventory
     int availableFuelBlock = gui.maininv.getItemPartialCount("Block");
     if (availableFuelBlock < 3)
      getfuel();

     // find one piece of fuel in the inventory
     WItem fuel = gui.maininv.getItemPartial("Block");
     if (fuel == null)
      continue;

     int fuelticks = 50; // it takes two blocks to fill the fire to 100

     // navigate to fire
     gui.map.pfRightClick(f, -1, 3, 0, null);
     try {
      gui.map.pfthread.join();
     } catch (InterruptedException e) {
      return;
     }

     if (terminate)
      return;
     
     try {
          Thread.sleep(TIMEOUT);
     } catch (InterruptedException e) {
          return;
     }
     
     Window cwnd = gui.getwnd("Fireplace");
     if (cwnd == null)
      continue;
     VMeter vm = cwnd.getchild(VMeter.class);
     if (vm == null)
      continue;

     gui.syslog.append("VMeter: "+vm.amount+" > "+(100-fuelticks),Color.CYAN);
     if (vm.amount > (100 - fuelticks))
      continue;

     int fueltoload = (100 - vm.amount) / fuelticks;
     
     // take fuel
     fuel.item.wdgmsg("take", new Coord(fuel.item.sz.x / 2, fuel.item.sz.y / 2));
     int timeout = 0;
     while (gui.hand.isEmpty()) {
      timeout += HAND_DELAY;
      if (timeout >= TIMEOUT)
       continue floop;
      try {
       Thread.sleep(HAND_DELAY);
      } catch (InterruptedException e) {
       return;
      }
     }
     gui.syslog.append("Hand is not empty!", Color.CYAN);
     for (; fueltoload > 0; fueltoload--) {
      if (terminate)
       return;
      while(!gui.hand.isEmpty()){
      gui.map.wdgmsg("itemact", Coord.z, f.rc.floor(posres), fueltoload == 1 ? 0 : 1, 0, (int) f.id,
        f.rc.floor(posres), 0, -1);
      try {
       Thread.sleep(1000);
      } catch (InterruptedException e) {
       return;
      }
      }
      timeout = 0;
      while (true) {
       WItem newfuel = gui.vhand;
       if (newfuel != fuel) {
        fuel = newfuel;
        break;
       }

       timeout += HAND_DELAY;
       if (timeout >= TIMEOUT)
        break;
       try {
        Thread.sleep(HAND_DELAY);
       } catch (InterruptedException e) {
        return;
       }
      }
     }

     WItem hand = gui.vhand;
     // if the fireplace is full already we'll end up with a
     // stockpile on the cursor
     if (hand != null) {
      gui.map.wdgmsg("place", Coord.z, 0, 3, 0);
      gui.map.wdgmsg("drop", hand.c.add(Inventory.sqsz.div(2)).div(Inventory.invsq.sz()));
     }
    }

   try {
    Thread.sleep(SLEEP);
   } catch (InterruptedException e) {
    return;
   }

   }

  }
 }
As one can see class implements Runnable, the same one we used to implement AutoKin in one of the previous posts (https://winter-in-hnh.blogspot.ru/2017/01/extending-haven-and-hearth-client.html) and as previously we create a run() method that actually implements what we want - lets have a look at what code does:

  1. We have a while loop that runs as long as terminate attribute is not set
  2. Every loop we write "Starting Loop!" into System chat so that we can identify loop running
  3. Then we have another loop going through all configured fireplaces, so next steps are executed for each fireplace
  4. Check availability of "Blocks" in the inventory and if we have less than three execute getfuel() method - we will add it later
  5. We define fuelticks to be 50 - since two blocks make it up for 100 in fuel meter
  6. Then we execute pfRightClick which invokes path finding to firm move to the object and then right-click it to open flower menu
  7. We wait for path finding thread to complete
  8. We check if we are to terminate
  9. We wait for TIMEOUT - while we wait another thread is going to react to opened flower menu selecting 'Open' - we will handle that later separately
  10. Search for window titled "Fireplace" and find a child element of that window of VMeter.class
  11. It's now good to drop a diagnostic message showing how much fuel we have in the Fireplace vs when we have to feed some fuel to it
  12. If we have enough fuel - switch to next Fireplace
  13. Take fuel - wood-block from inventory and wait till you get fuel in hand
  14. Once fuel is in hand - place it into Fireplace
  15. If fuel is left in hand - create a stockpile
  16. Sleep for SLEEP amount of seconds
  17. Repeat
So this implements our automation, what is left are various support functions, lets have a look at what they do
 private void getfuel() {
  GameUI gui = gameui();
  Glob glob = gui.map.glob;
  for (Gob s : stockpiles) {
   if (terminate)
    return;

   // make sure stockpile still exists
   synchronized (glob.oc) {
    if (glob.oc.getgob(s.id) == null)
     continue;
   }

   // navigate to the stockpile and load up on fuel
   gameui().map.pfRightClick(s, -1, 3, 1, null);
   try {
    gui.map.pfthread.join();
   } catch (InterruptedException e) {
    return;
   }

   // return if got enough fuel
   int availableFuelBlock = gui.maininv.getItemPartialCount("Block");
   if (availableFuelBlock >= 3)
    return;
  }
 }
Here we loop through all configured stockpiles right-clicking them to get whatever is stored in them, and once we have more fuel than expected (more than three) we complete the loop.
Next method is pretty important since that is what implements GobSelectCallback and allows to configure our fireplaces and stockpiles.
 public void gobselect(Gob gob) {
  Resource res = gob.getres();
  if (res != null) {
   if (res.name.equals("gfx/terobjs/pow")) {
    if (!fires.contains(gob)) {
     fires.add(gob);
     lblf.settext(fires.size() + "");
    }
   } else if (res.name.equals("gfx/terobjs/stockpile-wblock")) {
    if (!stockpiles.contains(gob)) {
     stockpiles.add(gob);
     lbls.settext(stockpiles.size() + "");
    }
   }
  }
 }
This method is called when you Alt+Click an object, what it does it checks object type by it's name and if it is a Fireplace or wood-block Stockpile object gets remembered in one of two arrays. Next two functions handle window closure or typeing while window is active:
 @Override
 public void wdgmsg(Widget sender, String msg, Object... args) {
  if (sender == cbtn)
   reqdestroy();
  else
   super.wdgmsg(sender, msg, args);
 }

 @Override
 public boolean type(char key, KeyEvent ev) {
  if (key == 27) {
   if (cbtn.visible)
    reqdestroy();
   return true;
  }
  return super.type(key, ev);
 }
Finally we have terminate method handling termination of automation activity
 public void terminate() {
  terminate = true;
  if (runner != null)
   runner.interrupt();
  this.destroy();
 }

Support and integration

Now we have all the functionality implemented and it's time to integrate it into the client, we also need a couple support things to make things happen. Since we are going to add a menu item button for our automation we will need to create a resource like described in one of the previous posts - https://winter-in-hnh.blogspot.ru/2017/01/lets-add-buttons.html
  1. Create a resource icon "res\paginae\amber\autofire" that would trigger "autofeedfire" action and register it in MenuGrid::attach() method
    p.add(paginafor(Resource.local().load("paginae/amber/autofire")));
    
  2. Register "autofeedfire" action in MenuGrid::use() method
            } else if (ad[1].equals("autofeedfire")){
             if (gui.getwnd("Auto Fire") == null) {
                    AutoFire sw = new AutoFire();
                    gui.map.autofire = sw;
                    gui.add(sw, new Coord(gui.sz.x / 2 - sw.sz.x / 2, gui.sz.y / 2 - sw.sz.y / 2 - 200));
                    synchronized (GobSelectCallback.class) {
                        gui.map.registerGobSelect(sw);
                    }
                }
            }
    
  3. Modify haven.MapView.canceltasks() method to  accommodate for autofire functionality
    if (autofire != null)
                autofire.terminate();
    
  4. Last thing to take care of would be to teach the client to automatically select 'Open' from flower menu when right-clicking the fireplace, this would require several changes:
    In Config.java we need to add autoopen paramater:
    public static boolean autoshear = Utils.getprefb("autoshear", false);
    public static boolean autoopen = Utils.getprefb("autoopen", false);
    

    In FloweMenu.java we need to ensure that if autoopen parameter is set it should be automatically picked:
    Config.autoshear && p.name.equals("Shear wool") ||
    Config.autoopen && p.name.equals("Open") ||
    p.name.equals(nextAutoSel) && System.currentTimeMillis() - nextAutoSelTimeout < 2000)) {
This should get everything in place - make sure to have RES file properly placed, rebuild the project and you should be all set.
This is how it should look like in game:

Done for today

So today we automated a task of feeding fireplaces, we used SteelRefueler as a template, added another button to the menu to trigger our functionality. Enjoy!

Keeping your Haven and Hearth client up to date

Every once in a while things get updated and after some time you might end up seeing something like:
Obviously you won't be able to login as long as your client is too old. And this is where we need to make sure we properly track upstream Amber client branch we forked.

How did we fall behind?

First of all - lets figure out how did it happen that we fell behind? To do so lets have a look at our GitHub repo (the one we created back here - https://winter-in-hnh.blogspot.ru/2017/01/haven-and-hearth-client.html when we forked Amber repository from romovs):
As you can see above our current branch is 45 commits behind master branch of romovs Amber repository. To see what has changed you can click on "Compare" button, make sure to select your fork as base, and romovs/amber as head, you should see something something like the following:
Here you can see the list of all 45 commits that we've missed so far, and in "Files changed" tab we will be able to see all  files that were changed. One thing to note - is that GitHub currently is not aware of changes we've done - all the changes we were doing so far (https://winter-in-hnh.blogspot.ru/2017/01/extending-haven-and-hearth-client.html and https://winter-in-hnh.blogspot.ru/2017/01/lets-add-buttons.html) were pretty much invisible for GitHub - we were doing all of them locally in our local git environment. So before we get into merging upstream changes, lets commit out own changes.

Commiting changes

To commit changes - right-click your project name in "Project Explorer" select "Team" and then "Commit":
This will bring up "Staging" tab - where you will be presented with changes that you've done so far - select all the files you've changed and write a commit message.
Make sure that you "Commit and Push" your changes - commit will only commit changes to your local repository, while Push, will also get them to remote - GitHub one. Once completed we should see our changes on GitHub:

Unfortunately we can no longer automatically merge, because our changes conflict with some of the changes done to upstream.

How do we merge?

Ok, so we are not at the stage when we would really want to push anything to Amber, so we'll need to change our changes to co-exist with what was modified. There are two GitHub articles that describe what we are going to do next:
So lets follow this guides. First of all lets check what our current local repository is bound to:
Nothing surprising here - this is the repository we created forking romovs/amber and cloned locally to build and modify our Haven and Hearth client (as described in https://winter-in-hnh.blogspot.ru/2017/01/haven-and-hearth-client.html). Now lets add an "upstream" repository:
With "upstream" set we can now fetch content from it:
Let's make sure we are on the "master" branch of our "origin":
Since we expect some merging problems lets use Eclipse IDE merging tool.
What we are planning to do, is to merge our Local repository and 'upstream/master' branch and then commit it to our remote 'origin/master', so first thing to do here is to configure what we will be merging:
So we have conflicts - conflicting places are marked in "Project Explorer" and can be identified in source by special tokens:
So lets start the "Merge tool" and fix our conflicts:
Once complete you should see a "Merged master" message next to your project name in "Project Explorer":
Now it's time to commit our merge the same way we commited our changes in the beginning of the article.

Observing consequencies

Once evertyhing is commited and pushed lets have a look at where we end up. On GitHub we now can compare romovs/amber repository as base to our repository as HEAD:

According to GitHub comparation we are two commits ahead - and that is exactly where we would want to be - ahead of our upstream. Lets try to compile the result.. and it works!

Well done!

If we don't frequently sync to upstreams - we fall behind and the time would come when a change is done to the client that obsoletes our working copy forcing us to re-sync. The further away we are from current state of upstream client - the more is probability that we'll have to merge conflicting changes - so we should be catching up with upstream as frequent as possible to avoid complex merges.

On high level merging is pretty simple:
  1. Make sure you have your upstream repository configured
  2. Fetch changes from your configured upstream repository
  3. Make sure you are on the "master" branch of your forked repository
  4. Merge the changes
  5. If you are lucky - all changes would be a simple fast-forward, if not - resolve conflicts
  6. Rebuild the project
  7. Enjoy







Tuesday 31 January 2017

Lets add buttons!

In my previous article (https://winter-in-hnh.blogspot.ru/2017/01/extending-haven-and-hearth-client.html) we extended Haven and Hearth client of our choice (Amber) with AutoKin functionality. We used GameUI::globtype() method to bind our AutoKin functionality to Control+K combination. This time we are going to try adding a menu button under Xtensions menu.




Placing icons into the menu

We'll start by adding a new icon into the menu - it's really easy to do, and requires addition of just one line. So lets have a look at ManuGrid.java class and it's implementation of attach() method.
    protected void attach(UI ui) {
        super.attach(ui);
        Glob glob = ui.sess.glob;
        synchronized (glob.paginae) {
            Collection<Pagina> p = glob.paginae;
            p.add(glob.paginafor(Resource.local().load("paginae/amber/coal11")));
            p.add(glob.paginafor(Resource.local().load("paginae/amber/coal12")));
            p.add(glob.paginafor(Resource.local().load("paginae/amber/branchoven")));
           // p.add(glob.paginafor(Resource.local().load("paginae/amber/steel")));
            p.add(glob.paginafor(Resource.local().load("paginae/amber/autosurvey")));
            p.add(glob.paginafor(Resource.local().load("paginae/amber/torch")));
            p.add(glob.paginafor(Resource.local().load("paginae/amber/clover")));
            p.add(glob.paginafor(Resource.local().load("paginae/amber/timers")));
            p.add(glob.paginafor(Resource.local().load("paginae/amber/autokin")));
        }
    }
Now we should have our icon in the menu, or maybe not? Lets build, run and have a look...

Crash and Burn!

Quiet expectidly client crashes with below exception:
java.lang.RuntimeException: Delayed error in resource paginae/amber/autokin (v-1), from local res source
 at haven.Resource$Pool$Queued.get(Resource.java:352)
 at haven.Resource$Pool$Queued.get(Resource.java:321)
 at haven.Glob$Pagina.res(Glob.java:150)
 at haven.Glob$Pagina.act(Glob.java:154)
 at haven.MenuGrid.cons(MenuGrid.java:93)
 at haven.MenuGrid.updlayout(MenuGrid.java:149)
 at haven.MenuGrid.tick(MenuGrid.java:433)
 at haven.Widget.tick(Widget.java:651)
 at haven.Widget.tick(Widget.java:651)
 at haven.GameUI.tick(GameUI.java:677)
 at haven.Widget.tick(Widget.java:651)
 at haven.UI.tick(UI.java:132)
 at haven.HavenPanel.run(HavenPanel.java:580)
 at java.lang.Thread.run(Unknown Source)
Caused by: haven.Resource$LoadException: Load error in resource paginae/amber/autokin(v-1), from local res source
 at haven.Resource$Pool.handle(Resource.java:409)
 at haven.Resource$Pool.access$1100(Resource.java:298)
 at haven.Resource$Pool$Loader.run(Resource.java:550)
 ... 1 more
 Suppressed: haven.Resource$LoadException: Load error in resource paginae/amber/autokin(v-1), from filesystem res source (res)
  ... 4 more
 Caused by: java.io.FileNotFoundException: res\paginae\amber\autokin.res (The system cannot find the path specified)
  at java.io.FileInputStream.open0(Native Method)
  at java.io.FileInputStream.open(Unknown Source)
  at java.io.FileInputStream.<init>(Unknown Source)
  at haven.Resource$FileSource.get(Resource.java:190)
  at haven.Resource$Pool.handle(Resource.java:393)
  ... 3 more
Caused by: java.io.FileNotFoundException: Could not find resource locally: paginae/amber/autokin
 at haven.Resource$JarSource.get(Resource.java:202)
 at haven.Resource$Pool.handle(Resource.java:393)
 ... 3 more
I guess I will post more on exception debugging in Haven and Hearth client at some point later, but now what is immediately evident from this exception is that it wasn't able to find a resource associated with something in MenuGrid and it's definitely related to our change since we have "autokin" all over the place. Solution to our problem is right in front of us:
Caused by: java.io.FileNotFoundException: res\paginae\amber\autokin.res (The system cannot find the path specified)
Which corresponds to our code looking like this:
p.add(glob.paginafor(Resource.local().load("paginae/amber/autokin")));
So we have instructed a menu item to be added to the MenuGrid using autokin icon, which is translated to local path "res\paginae\amber\autokin.res". By the looks of it path is relative to current directory - directory from where the client was started. But where do we get a RES file?

Lets mine some buttons

There are several utilities that allow RES file creation to name a few:
I had some poor experience with Boshaw's Hafen Layer Utility and ended-up using his LayerUtil the Java version - it worked really well and that is what we will use for this post. It can be compiled using the same approach as described in this post - https://winter-in-hnh.blogspot.ru/2017/01/haven-and-hearth-client.html.
But before we get into that, lets start by simply duplicating one of existing RES files. In your "build" folder locate "amber-res.jar" and open it with your favourite Zip archiver - I'll be using 7Zip:
Lets extract "amber.res" and rename it to "autokin.res" placing it under "res\paginae\amber\" directory relative to our "build". Like this:
This should solve the problem of non-existent file and we are still sure that our code does the right thing - so no changes needed and thus no recompilation required, lets start the client... again.

Expect the unexpected

This time client starts without any problem, we get in the game and get a new button:
Not exactly what I would expect, but kina makes sense - RES file is not just an image, but rather a compiled resource. This means we need our own resource file that we would be able to use for our purpose. So lets decode one file as a template. Since we want our icon to be "inside Xtensions" lets pick one of the resources for menu items placed there - so go ahead and unpack another RES file. I will be using coal12.res. 
Ok, so now you should have your "template" extracted and LayerUtil downloaded and built somewhere. I created a project for it in the same workspace, so it is placed next to my amber project directory:
Now lets decode coal12.res and finally have a look at what is inside:

Running this you should end up having a "dout" folder with several files and folders in it looking like the following:
First of all lets change the image associated with the resource, I'll use something like:
Put it into "image" folder as "image_0.png". In "pagina" folder we have "pagina_0.data" which contains description for our menu item, so lets change it to "Auto add Kin's from secrets.txt file." here is how it looks like in my Notepad++:
Now the most challanging part - modifying the "action"; in "action" folder modify "action_0.data" file to look like the following:

So I changed the name to be "Winter AutoKin", ad length to be "2" - used to be "3", and set ad[1] to "autokin". Great, now lets recompile our resource.
Now we have our recompiled coal12.res - lets rename it to "autokin.res" and replace our previous version in "res\paginae\amber\", and lets start the client:
So we have successfully created our button with name, description, image and action associated with it. The part missing now is binding the button to functionality - so lets go ahead and back to code.

The binding of the button

To bind the button we need to add it's action into MenuGrid's use() method, there are two use() methods for MenuGrid class - we need the public one that takes array of String's as an argument:
    public void use(String[] ad) {
There is a big IF block describing all known actions and their arguments, we will add our action at the bottom of the list:
        } else if (ad[1].equals("clover")) {
            Thread t = new Thread(new FeedClover(gui), "FeedClover");
            t.start();
        } else if (ad[1].equals("autokin")){
         Thread t = new Thread(new AutoKin(gui), "AutoKin");
         t.start();
        }
    }

As you can see this is pretty much the same as in https://winter-in-hnh.blogspot.ru/2017/01/extending-haven-and-hearth-client.html where we did exactly the same for a key chord - create a new thread with AutoKin class as an argument and start the thread.
Now, lets rebuild and start the client hopefully for the last time in this post.

...and it works!

Recap

Honestly speaking it feels like adding a button in Haven and Hearth client is a bit overcomplicated, resource compilation in particular is not as straightforward and clear, but I guess developers had a reason to do it so. Looking back at the post this is what we've done to make it all happen:
  1. Prepare resource file by decoding an existing pagina resource as a template, modifying "image", "pagina" and "action" content, encoding it into a new file and placing it into appropriate local directory
  2. Load the resource file created from within MenuGrid.java class attach() method 
  3. Modify MenuGrid.java class use() method to utilize your ad's defined in "action" in step 1
Now we are able to create menu items and bind functionality to them! Good luck!

Monday 30 January 2017

Extending Haven and Hearth Client

Hey All, in my previous post - https://winter-in-hnh.blogspot.ru/2017/01/haven-and-hearth-client.html we explored Amber - an advanced Haven and Hearth client by romovs; we learned to fork it on GitHub, clone it to a local repository and compile it in Eclipse IDE, so now it's time for a change, or actually to make a change.

What's the plan?

This particular amendment will allow us to simplify adding Kin's into our Kin list - while adding somebody via hearth secret is not hard and is easily manageble for a solo player, or a player with a couple of friends it gets really challanging once you have over 100 hearth secrets which would be the case for a bigger village or even kingdom. So what are we going to do? We are going to create an AutoKin class which sole purpose would be to open a text file, read hearth secrets from it and tell the client to add Kin's with respective secrets.

Setting everything up

But before we get to coding there is one additional step we need to perform and that is telling Eclipse IDE that we are actually dealing with a Java project. Previously (in https://winter-in-hnh.blogspot.ru/2017/01/haven-and-hearth-client.html) we created a generic project to store our source code, and then only used Eclipse IDE to check out source code from Git and told it to execute an Ant build using build.xml supplied by the project. Since now we are getting into more advanced stuff we need to make sure Eclipse IDE has an idea of what we are working with. To do so right-click your project name in Project Explorer tab and select Properties. Then click "Project Facets" and select "Java" to instruct Eclipse IDE that we are going to be using Java in our project.
Now we are all set to code.

Adding AutoKin

In our Project Explorer tab - lets pick a nice place for where we will put our AutoKin class, to me the most obvious place is with other automation tasks:

So right-click "haven.automation" folder and select "New > Class":

Now select the name of your class and check the package where it will be added, as well as pick interfaces it is going to implement:
It's worth noting that Runnable is a standard Java class (https://docs.oracle.com/javase/8/docs/api/java/lang/Runnable.html) that needs to implement a single method - run().

Once finished you should see your newly added AutoKin.java in Project Explorer window under "haven.automation" package in the source tree:
As you can see Eclipse IDE also prepopolated our class based on information we provide, so it "implements Runnable" and has stub run() method.


Coding AutoKin

Now that we successfully added a new class lets populate it with some actual functionality.

package haven.automation;
import java.awt.Color;
import java.io.BufferedReader;
import java.io.IOException;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;

import haven.GameUI;

public class AutoKin implements Runnable {
  // private attribute pointing to GameUI
  private GameUI gui;
  // AutoKin constructor
  public AutoKin(GameUI gui) {
    this.gui = gui;
  }
  @Override
  public void run() {
    gui.syslog.append("AutoAdding Kin's:", Color.WHITE);
    Path file = Paths.get("secrets.txt");
    Charset charset = Charset.forName("UTF-8");
    try (BufferedReader reader = Files.newBufferedReader(file, charset)) {
      String line = null;
      while ((line = reader.readLine()) != null) {
        gui.syslog.append(line,Color.BLUE);
        gui.wdgmsg(gui.buddies,"bypwd",line);
      }
    } catch (IOException x) {
      gui.syslog.append("Exception: "+x.toString(),Color.RED);
    }
  }
}

Now, this is pretty much self explanatory - very basic stuff. We create a Path pointing to "secrets.txt" file in current directory (directory from where the client was started) and try to read it as if it was "UTF-8" encoded line by line. Every line is interpreted as a hearth secret, so we try to add a kin (buddy) using their secret (bypwd). This is good time to rebuild the client and see if we broke anything with our addition. I hope you remember how to execute the Ant build - so right-click on "build.xml", and then select "Run As > Ant Build" and there it goes - remember to save your changes! In Console tab you should be able to see your project building:
Buildfile: C:\workspace\amber\build.xml
build-env:
hafen-client:
    [javac] Compiling 1 source file to C:\workspace\amber\build\classes
buildinfo:
lib-classes:
jar:
      [jar] Updating jar: C:\workspace\amber\build\hafen.jar
res-jar:
l10n-jar:
jars:
deftgt:
BUILD SUCCESSFUL
Total time: 4 seconds

Great, so our functionality compiled, but how do we trigger it?

Lets run() the Runnable

For now we are not going to get into intricacies of menu's, windows, widgets and buttons, but rather go the easy way and bind our functionality to a hotkey. To do so, we will modify globtype() method in GameUI.java class. GameUI.class is located under package "haven" and particular place within globtype() we are interested in is the following:
        } else if (!ev.isShiftDown() && ev.getKeyCode() == KeyEvent.VK_Q) {
            Thread t = new Thread(new PickForageable(this), "PickForageable");
            t.start();
            return true;
        } else if (ev.isControlDown() && ev.getKeyCode() == KeyEvent.VK_K){
         // AutoKin
         Thread t = new Thread(new AutoKin(this), "AutoKin");
         t.start();
         return true;
        }
        return (super.globtype(key, ev));
As you can see this method takes two arguments - character representing key pressed and a KeyEvent class, we are using the latter since we need slightly more information than what is the key character. Above you can see us adding another clause binding execution of AutoKin in a new thread to Control+K chord.

Auto-adding-Kins

With all changes done and saved, we recompile our build and start the client. Once logged in and in game lets press Control+K for the fun of it:
Yay! Our code picked up and tried opening secrets.txt, which was not there and thus failed with NoSuchFileException. Now let's create "secrets.txt", which will only consist of single line "A9XylWM6" and put it into our "build" directory - and now we are all set to press Control+K again.
Ok, now this is better - we see the hearth secret we are trying to add, but we see that nobody has it - this happens because "l" and "I" look alike, so we just had the wrong hearth secret - lets fix it in the file. This time no red line appears and "Kith and Kin" window pops-up showing our new added Kin(s).

Achievement Unlocked

Here is what we have achieved:
  1. Set our project to be Java based
  2. Developed AutoKin functionality - reading hearth secrets from file and adding kins by their hearth secret
  3. Added a hotkey to invoke our AutoKin functionality
  4. Compiled updated Haven and Hearth client



Sunday 29 January 2017

Haven and Hearth Client

To start doing arcane magic in Haven and Hearth one needs to learn to checkout, change and compile Haven and Hearth client. One of the most advanced clients for Haven and Hearth is developed by romovs and is called Amber, and that's what we would be using in our journey.

First things first

To begin with we need to ensure that we have all basic things that would be needed in future:
  • Java Development Kit (JDK) - make sure to have latest JDK installed, you can download it from http://www.oracle.com/technetwork/java/javase/downloads/index.html 
  • GIT client - GIT is version control system. Since we will be using GitHub to get the source code, we need a client. You can still do without GIT client and download ZIPed snapshots but you won't be able to synchronize your changes to GitHub then. You can get GIT client from https://git-scm.com/downloads
  • Integrated Development Environment (IDE) - we will be using Eclipse IDE, there are multiple other options IDEA, NetBeans to name a few. There is no particular reason why we are going to be using Eclipse, approach would be the same whatever you decide to pick. Eclipse can be downloaded here - https://eclipse.org/downloads/eclipse-packages/
Now we should be ready to get the source...

Use the source

To begin with lets fork Amber on GitHub, you can do so by clicking "Fork" button in the upper right corner of Amber repository page - https://github.com/romovs/amber:
This should create a copy of Amber repository for yourself and you should see something like this on repository page created for your copy:
Congratulations now you have your very own copy of Amber hosted on GitHub! Now lets download it and start using it. Time to start our IDE - after configuring workspace (directory where IDE will store your project files) you get to Welcome screen, where we will select "Check out projects from Git":
This presents us with a choice to use an existing local repository or to clone a URI, since we don't yet have any local repositories second options is what we are after:
You will be presented with the following form to specify Location of your repository and credentials to be used to access it. Once you fill in URI - Host and Repository path will be filled in automatically, authenitcation credentials are ones you use to access your GitHub account:
Next screen allows to select branches we want to check out - we are only interested in "master" branch, so that's what we select:
Now we need to specify where to store the copy localy and which branch is to be used:
Note that we checked "Clone submodules" - this is not really needed currently for Amber, but you never know what will change and perhaps at some point Amber might have external dependencies. Now Eclipse will clone the repository and place a local copy whereever you pointed it to. This gets us to next step - creating a project to work with checked out sources. Since source have no project files we will be importing them as "general project":
Next screen will allow you to set the project name and you are done. Now you have your GitHub repository cloned to your local PC and stored under directory that you specified and "Project Explorer" tab should be looking like the following:
Amber is built using Apache Ant, luckily enough it is integrated into Eclipse IDE and there is no need to download it separately and "build.xml" is configuration file defining build targets (like Makefile for those of you who are more familiar with "make". So we right-click "build.xml" and select "Run As -> Ant Build":
Follow Console tab for any erros, a couple of tasks need to be performed and the line we are looking for is "BUILD SUCCESSFUL":
Yay! Now lets go to workspace and see what we have there:

Note the "build" folder - that is where all the compiled classes went, but before that lets copy "run.bat" from "etc" folder - it contains startup parameters for the client. So here is how the final state of "build" folder should look like from inside:
 
Now lets start "run.bat" and behold our glorious creation.

The Deed is Done

So to recap of what we have done:
  1. We setup our development environment comprised of JDK, Eclipse IDE and Git setup
  2. We forked Amber official repository on GitHub creating a copy for ourselves
  3. We cloned our new repository to a local copy to work on it
  4. We built Amber client using build.xml supplied with the project
  5. We copied run.bat supplied with the project to build directory created during build process
  6. We executed run.bat to get client running

This should get us rolling and set a path for us to expore Artifice and Arcana of Haven and Hearth world.