Tuesday 7 February 2017

Chop the head off - making a headless Haven and Hearth client

Today I figured out I need to cut the head off Amber Haven and Hearth client. Main reason for it is that JOGL doesn't really like raspberry pi - there are lots of forum discussions about raspberry pi and JOGL, a couple suggested solutions, but unfortunately none of them really worked for me. So what we are going to try to do today is introduce a way to start Haven and Hearth client without visible window and lets hope it solves the problem.

Find the neck

First of all we need to identify places we need to modify in order to supress window creation. This should be a pretty simple process - we build our client on raspberry pi and run it:
This gives us two places in MainFrame.java where we crash, so lets move them under a condition, which would be a command line option, so we'll get something like:
        add(p);
        if (!Config.headless)
        	pack();
        setResizable(!Utils.getprefb("wndlock", false));
        p.requestFocus();
        seticon();
        if (!Config.headless)
        	setVisible(true);
        p.init();
        addWindowListener(new WindowAdapter() {

Obviously we need to add "headless" as a field in Config.java and modify Config.cmdline() to set that field.
public class Config {
...
	public static boolean headless = false;
...
    public static void cmdline(String[] args) {
        PosixArgs opt = PosixArgs.getopt(args, "hdPGp:U:r:A:u:C:l:");
...
                case 'l':
                	headless = true;
                	System.out.println("Running Headless - enjoy!");
                	break;
...
Ok, now we have an option lets compile and try to use it, we might want to modify run.bat at some point, but for now lets just run the client from command line manually.
java -Xms512m -Xmx1024m -jar hafen.jar -U http://game.havenandhearth.com/hres/ game.havenandhearth.com -l true
This unfortunately still doesn't give us a stable state:
So, we'll get this exception under condition as well, like this:
        public void run() {
            try {
                uglyjoglhack();
                if (state == null){
                	if(!Config.headless)
                          throw (new RuntimeException("State applier is still null after redraw"));
                	else
                	  System.out.println("State applier is still null after redraw");
                	
                }

Now we got our headless client running, but it is still pretty useless, since there is no we can interact with it.

I have no mouth and I have to scream

So, we need a way to tell client what we want from it without interacting with graphical interface, so lets create a console interface! Our console interface is going to be pretty simple - it will read a command from standard input and will pass it to root console:
package haven.automation;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;

import haven.HavenPanel;

public class ConsoleInterface implements Runnable {

	@Override
	public void run() {
		// TODO Auto-generated method stub
		BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
		String cmd = null;
		while(true){
			System.out.print("?> ");
			try {
				cmd = br.readLine();
			} catch (IOException e) {
				// TODO Auto-generated catch block
				e.printStackTrace();
			}
			System.out.println("< "+cmd);
			try {
				HavenPanel.lui.root.ui.cons.run(cmd);
			} catch (Exception e) {
				// TODO Auto-generated catch block
				e.printStackTrace();
			}
		}

	}

}
We need to remember to start our ConsoleInterface thread, so something like this:
...
        this.g = new ThreadGroup(HackThread.tg(), "Haven client");
        this.mt = new HackThread(this.g, this, "Haven main thread");
        this.ct = new Thread(new ConsoleInterface(),"Haven Console Interface");
        this.ct.start();
...
Now lets try it out - we will be using our console interface together with graphical interface for demonstration purposes.

Ways to go

This enables us with a number of interesting further developments - next thing we should do is enable our ConsoleInterface to receive command output, and then extend it with additional commands to interact with objects and subjects.


Updating JOGL to 2.3.2 in Amber - Haven and Hearth Client because we can

Today I was looking at Amber Haven and Hearth client and thing I noticed was that it was running some really old JOGL (library used to do graphics - OpenGL). So I thought - why would I want to run something from 2013 - most likely JOGL supplied with Amber is 2.1.2 released 1 November 2013.

Get your libraries

Amber has JOGL libraries predownloaded into "lib" directory and uses slightly different naming notation than the actual JOGL release.
So we need to update "gluegen-rt.jar" and "jogl.jar" in lib folder and then download native library jar's into "jogl-natives" folder.  Note that on the above screenshot we already have updated libraries, plus I downloaded armv6hf since I am playing with Haven and Hearth client on Raspberry Pi. Once we identified which libraries we need we can download them from https://jogamp.org/deployment/v2.3.2/

A couple of things worth mentioning - as stated above JOGL release uses a slightly different naming scheme, so "jogl.jar" in Amber client is "jogl-all.jar" in JOGL release, and "jogl-natives-X-Y.jar" on Amber side matches "jogl-all-natives-X-Y.jar" on JOGL release side.

There are several pitfalls that we need to avoid:
  1. All libraries should be from the same version - we can not use library of one release and native library from another that would cause a "Certificate mismatch" exception when trying to run the client
  2. Naming convention should be intact - build.xml expects particular jar names:

  3. jogl-jar and gluegen.jar can come double compressed, so you might need to extract jar from jar:

Fixing errors

Once we are done updating libraries and try to compile we are going to get multiple errors majority of which are going to be of two types:
    [javac] C:\workspace\amber\src\haven\HavenPanel.java:43: error: package javax.media.opengl does not exist
    [javac] import javax.media.opengl.*;
    [javac] ^
    [javac] C:\workspace\amber\src\haven\HavenPanel.java:71: error: cannot find symbol
    [javac]     private static GLCapabilities stdcaps() {
    [javac]                    ^
    [javac]   symbol:   class GLCapabilities
    [javac]   location: class HavenPanel
We are going to fix those that are about packages that do not exist. Reason for those errors is that in new libraries (not sure when the change happened, but who cares and why?) have "opengl" moved from "javax.media" package to "com.jogamp".

Once all packages are fixed there are two more things to be addressed. One of them are screenshots, it happens so, that com.jogamp.opengl.util.awt.Screenshot was deprecated in JOGL 2.x and is completely removed in 2.3.2. To address this we will have to modify "takescreenshot()" method defined within an anonymous GLEventListener object in initgl() method in HavenPanel.

            public void takescreenshot(GLAutoDrawable d) {
                try {
                    String curtimestamp = new SimpleDateFormat("yyyy-MM-dd_HH.mm.ss.SSS").format(new Date());
                    File outputfile = new File(String.format("screenshots/%s.png", curtimestamp));
                    outputfile.getParentFile().mkdirs();
                    AWTGLReadBufferUtil glReadBufferUtil=new AWTGLReadBufferUtil(d.getGLProfile(),false);
                    BufferedImage image=glReadBufferUtil.readPixelsToBufferedImage(d.getGL(),true);
                    ImageIO.write(image, "png", outputfile);                  
                    ui.root.findchild(GameUI.class).msg(String.format("Screenshot has been saved as \"%s\"", outputfile.getName()), Color.WHITE);
                } catch (Exception ex) {
                    System.out.println("Unable to take screenshot: " + ex.getMessage());
                }
            }

            public void display(GLAutoDrawable d) {
                if (HavenPanel.needtotakescreenshot) {
                    takescreenshot(d);
                    HavenPanel.needtotakescreenshot = false;
                }

As you can see, we no longer pass width and height into functional and we use AWTGLReadBufferUtil to get image data and then save it into the file via ImageIO.write().

This should leave us with two issues in GLProgram.create() and GLShadre.create() methods. Root cause of these issues is that gl.glCreateProgramObjectARB() and gl.glCreateShaderObjectARB() no longer return and Integer type, but Long type - so we'll cast result to Integer - this might not be perfect and might cause some issues going forward, but that seems to be the easiest fix, since changing type of id to Long is causing further incompatibilities.

Time to run

Ok, now we finally should be able to compile our code with new libraries and enjoy Haven and Hearth Client with JOGL 2.3.2

Thursday 2 February 2017

Setting things on Fire! ...in an automatic way

One of the benefits of having access to Haven and Hearth client source code is ability to automate certain routine tasks - to make them less routine or more efficient or both. So why don't we try automating something?

Task Definition

We have Fireplaces and Stockpiles - Fireplaces burn, Stockpiles store fuel (wood blocks) for our Fireplaces. We want to make sure that we have a couple of wood-blocks in inventory (no less than 3), we go to Fireplaces one by one, click-Open them, check what is the fuel level and if it is less than 50 - half, then we drop another wood block into the Fireplace. Relax for half an hour - repeat.

Implementation

We'll base our implementation on code that already exists (SteelRefueler.java) and will modify it for our use-case. So lets create a new class under "haven.automation" extending haven.Window and implementing haven.automation.GobSelectCallback
public class AutoFire extends Window implements GobSelectCallback {
 private static final Text.Foundry infof = new Text.Foundry(Text.sans, 10).aa(true);
 private static final Text.Foundry countf = new Text.Foundry(Text.sans.deriveFont(Font.BOLD), 12).aa(true);
 private List<Gob> fires = new ArrayList<>();
 private List<Gob> stockpiles = new ArrayList<>();
 private final Label lblf, lbls;
 public boolean terminate = false;
 private Button clearbtn, runbtn, stopbtn;
 private static final int TIMEOUT = 2000;
 private static final int HAND_DELAY = 8;
 private static final int SLEEP = 30 * 60 * 1000; // 30 min
 private Thread runner;
Lets have a look at what we do here:

  • infof - foundry for information labels
  • countf - foundry for counter labels
  • fires - array of graphic objects representing fireplaces
  • stockpiles - array of graphic objects representing stockpiles
  • lblf, lbls - labels to show number's of fireplaces and stockpiles configured
  • terminate - a terminate flag
  • clearbton, runbtn, stopbtn - objects for "Clear", "Run" and "Stop" buttons
  • TIMEOUT - two seconds of delay between actions that need to complete before the next action
  • HAND_DELAY - eight milliseconds of delay between attempts to use item in hand
  • SLEEP - delay between loops
  • runner - background thread running our automation
Now lets write a constructor for our class:
 public AutoFire() {
  super(new Coord(270, 180), "Auto Fire");

  final Label lbl = new Label("Alt + Click to select fireplaces and stockpiles.", infof);
  add(lbl, new Coord(30, 20));

  Label lblctxt = new Label("Fires Selected:", infof);
  add(lblctxt, new Coord(15, 60));
  lblf = new Label("0", countf, true);
  add(lblf, new Coord(110, 58));

  Label lblstxt = new Label("Stockpiles Selected:", infof);
  add(lblstxt, new Coord(135, 60));
  lbls = new Label("0", countf, true);
  add(lbls, new Coord(235, 58));

  clearbtn = new Button(140, "Clear Selection") {
   @Override
   public void click() {
    fires.clear();
    stockpiles.clear();
    lblf.settext(fires.size() + "");
    lbls.settext(stockpiles.size() + "");
   }
  };
  add(clearbtn, new Coord(65, 90));

  runbtn = new Button(140, "Run") {
   @Override
   public void click() {
    if (fires.size() == 0) {
     gameui().error("No fires selected.");
     return;
    } else if (stockpiles.size() == 0) {
     gameui().error("No stockpiles selected.");
     return;
    }

    this.hide();
    cbtn.hide();
    clearbtn.hide();
    stopbtn.show();
    terminate = false;

    runner = new Thread(new Runner(), "Auto Fire");
    runner.start();
   }
  };
  add(runbtn, new Coord(65, 140));

  stopbtn = new Button(140, "Stop") {
   @Override
   public void click() {
    terminate = true;
    // TODO: terminate PF
    this.hide();
    runbtn.show();
    clearbtn.show();
    cbtn.show();
   }
  };
  stopbtn.hide();
  add(stopbtn, new Coord(65, 140));
 }

So what is happening here? We create our window, call super() constructor setting it's coordinates and title to "Auto Fire". Then we create labels with respective content and place them around the window. Once that is done - we add our "Clear Selection", "Run" and "Stop" buttons placing them in the window and hiding the "Stop" button.

Each button has click() method overriden with functionality we want it to expose:

  • "Clear Selection" button would empty fires and stockpiles arrays and update text content of labels with appropriate numbers corresponding to new sizes of said array
  • "Run" button will trigger a check for fires and stockpiles arrays sizes and will present an error in case either of two is empty, then it will hide other buttons and show the "Stop" one and create a thread running automation logic
  • "Stop" button sets termnination attribute, hides itself and shows other buttons
As one can see constructor is pretty simple and revolves around setting up the window, defining labels and buttons and associating functionality with button clicks.


Now lets write the actual automation:
 private class Runner implements Runnable {

  @Override
  public void run() {
   GameUI gui = gameui();
   while (!terminate) {
    gui.syslog.append("Starting Loop!",Color.CYAN);
    floop: for (Gob f : fires) {
     // take fuel from stockpiles if we don't have enough in the
     // inventory
     int availableFuelBlock = gui.maininv.getItemPartialCount("Block");
     if (availableFuelBlock < 3)
      getfuel();

     // find one piece of fuel in the inventory
     WItem fuel = gui.maininv.getItemPartial("Block");
     if (fuel == null)
      continue;

     int fuelticks = 50; // it takes two blocks to fill the fire to 100

     // navigate to fire
     gui.map.pfRightClick(f, -1, 3, 0, null);
     try {
      gui.map.pfthread.join();
     } catch (InterruptedException e) {
      return;
     }

     if (terminate)
      return;
     
     try {
          Thread.sleep(TIMEOUT);
     } catch (InterruptedException e) {
          return;
     }
     
     Window cwnd = gui.getwnd("Fireplace");
     if (cwnd == null)
      continue;
     VMeter vm = cwnd.getchild(VMeter.class);
     if (vm == null)
      continue;

     gui.syslog.append("VMeter: "+vm.amount+" > "+(100-fuelticks),Color.CYAN);
     if (vm.amount > (100 - fuelticks))
      continue;

     int fueltoload = (100 - vm.amount) / fuelticks;
     
     // take fuel
     fuel.item.wdgmsg("take", new Coord(fuel.item.sz.x / 2, fuel.item.sz.y / 2));
     int timeout = 0;
     while (gui.hand.isEmpty()) {
      timeout += HAND_DELAY;
      if (timeout >= TIMEOUT)
       continue floop;
      try {
       Thread.sleep(HAND_DELAY);
      } catch (InterruptedException e) {
       return;
      }
     }
     gui.syslog.append("Hand is not empty!", Color.CYAN);
     for (; fueltoload > 0; fueltoload--) {
      if (terminate)
       return;
      while(!gui.hand.isEmpty()){
      gui.map.wdgmsg("itemact", Coord.z, f.rc.floor(posres), fueltoload == 1 ? 0 : 1, 0, (int) f.id,
        f.rc.floor(posres), 0, -1);
      try {
       Thread.sleep(1000);
      } catch (InterruptedException e) {
       return;
      }
      }
      timeout = 0;
      while (true) {
       WItem newfuel = gui.vhand;
       if (newfuel != fuel) {
        fuel = newfuel;
        break;
       }

       timeout += HAND_DELAY;
       if (timeout >= TIMEOUT)
        break;
       try {
        Thread.sleep(HAND_DELAY);
       } catch (InterruptedException e) {
        return;
       }
      }
     }

     WItem hand = gui.vhand;
     // if the fireplace is full already we'll end up with a
     // stockpile on the cursor
     if (hand != null) {
      gui.map.wdgmsg("place", Coord.z, 0, 3, 0);
      gui.map.wdgmsg("drop", hand.c.add(Inventory.sqsz.div(2)).div(Inventory.invsq.sz()));
     }
    }

   try {
    Thread.sleep(SLEEP);
   } catch (InterruptedException e) {
    return;
   }

   }

  }
 }
As one can see class implements Runnable, the same one we used to implement AutoKin in one of the previous posts (https://winter-in-hnh.blogspot.ru/2017/01/extending-haven-and-hearth-client.html) and as previously we create a run() method that actually implements what we want - lets have a look at what code does:

  1. We have a while loop that runs as long as terminate attribute is not set
  2. Every loop we write "Starting Loop!" into System chat so that we can identify loop running
  3. Then we have another loop going through all configured fireplaces, so next steps are executed for each fireplace
  4. Check availability of "Blocks" in the inventory and if we have less than three execute getfuel() method - we will add it later
  5. We define fuelticks to be 50 - since two blocks make it up for 100 in fuel meter
  6. Then we execute pfRightClick which invokes path finding to firm move to the object and then right-click it to open flower menu
  7. We wait for path finding thread to complete
  8. We check if we are to terminate
  9. We wait for TIMEOUT - while we wait another thread is going to react to opened flower menu selecting 'Open' - we will handle that later separately
  10. Search for window titled "Fireplace" and find a child element of that window of VMeter.class
  11. It's now good to drop a diagnostic message showing how much fuel we have in the Fireplace vs when we have to feed some fuel to it
  12. If we have enough fuel - switch to next Fireplace
  13. Take fuel - wood-block from inventory and wait till you get fuel in hand
  14. Once fuel is in hand - place it into Fireplace
  15. If fuel is left in hand - create a stockpile
  16. Sleep for SLEEP amount of seconds
  17. Repeat
So this implements our automation, what is left are various support functions, lets have a look at what they do
 private void getfuel() {
  GameUI gui = gameui();
  Glob glob = gui.map.glob;
  for (Gob s : stockpiles) {
   if (terminate)
    return;

   // make sure stockpile still exists
   synchronized (glob.oc) {
    if (glob.oc.getgob(s.id) == null)
     continue;
   }

   // navigate to the stockpile and load up on fuel
   gameui().map.pfRightClick(s, -1, 3, 1, null);
   try {
    gui.map.pfthread.join();
   } catch (InterruptedException e) {
    return;
   }

   // return if got enough fuel
   int availableFuelBlock = gui.maininv.getItemPartialCount("Block");
   if (availableFuelBlock >= 3)
    return;
  }
 }
Here we loop through all configured stockpiles right-clicking them to get whatever is stored in them, and once we have more fuel than expected (more than three) we complete the loop.
Next method is pretty important since that is what implements GobSelectCallback and allows to configure our fireplaces and stockpiles.
 public void gobselect(Gob gob) {
  Resource res = gob.getres();
  if (res != null) {
   if (res.name.equals("gfx/terobjs/pow")) {
    if (!fires.contains(gob)) {
     fires.add(gob);
     lblf.settext(fires.size() + "");
    }
   } else if (res.name.equals("gfx/terobjs/stockpile-wblock")) {
    if (!stockpiles.contains(gob)) {
     stockpiles.add(gob);
     lbls.settext(stockpiles.size() + "");
    }
   }
  }
 }
This method is called when you Alt+Click an object, what it does it checks object type by it's name and if it is a Fireplace or wood-block Stockpile object gets remembered in one of two arrays. Next two functions handle window closure or typeing while window is active:
 @Override
 public void wdgmsg(Widget sender, String msg, Object... args) {
  if (sender == cbtn)
   reqdestroy();
  else
   super.wdgmsg(sender, msg, args);
 }

 @Override
 public boolean type(char key, KeyEvent ev) {
  if (key == 27) {
   if (cbtn.visible)
    reqdestroy();
   return true;
  }
  return super.type(key, ev);
 }
Finally we have terminate method handling termination of automation activity
 public void terminate() {
  terminate = true;
  if (runner != null)
   runner.interrupt();
  this.destroy();
 }

Support and integration

Now we have all the functionality implemented and it's time to integrate it into the client, we also need a couple support things to make things happen. Since we are going to add a menu item button for our automation we will need to create a resource like described in one of the previous posts - https://winter-in-hnh.blogspot.ru/2017/01/lets-add-buttons.html
  1. Create a resource icon "res\paginae\amber\autofire" that would trigger "autofeedfire" action and register it in MenuGrid::attach() method
    p.add(paginafor(Resource.local().load("paginae/amber/autofire")));
    
  2. Register "autofeedfire" action in MenuGrid::use() method
            } else if (ad[1].equals("autofeedfire")){
             if (gui.getwnd("Auto Fire") == null) {
                    AutoFire sw = new AutoFire();
                    gui.map.autofire = sw;
                    gui.add(sw, new Coord(gui.sz.x / 2 - sw.sz.x / 2, gui.sz.y / 2 - sw.sz.y / 2 - 200));
                    synchronized (GobSelectCallback.class) {
                        gui.map.registerGobSelect(sw);
                    }
                }
            }
    
  3. Modify haven.MapView.canceltasks() method to  accommodate for autofire functionality
    if (autofire != null)
                autofire.terminate();
    
  4. Last thing to take care of would be to teach the client to automatically select 'Open' from flower menu when right-clicking the fireplace, this would require several changes:
    In Config.java we need to add autoopen paramater:
    public static boolean autoshear = Utils.getprefb("autoshear", false);
    public static boolean autoopen = Utils.getprefb("autoopen", false);
    

    In FloweMenu.java we need to ensure that if autoopen parameter is set it should be automatically picked:
    Config.autoshear && p.name.equals("Shear wool") ||
    Config.autoopen && p.name.equals("Open") ||
    p.name.equals(nextAutoSel) && System.currentTimeMillis() - nextAutoSelTimeout < 2000)) {
This should get everything in place - make sure to have RES file properly placed, rebuild the project and you should be all set.
This is how it should look like in game:

Done for today

So today we automated a task of feeding fireplaces, we used SteelRefueler as a template, added another button to the menu to trigger our functionality. Enjoy!

Keeping your Haven and Hearth client up to date

Every once in a while things get updated and after some time you might end up seeing something like:
Obviously you won't be able to login as long as your client is too old. And this is where we need to make sure we properly track upstream Amber client branch we forked.

How did we fall behind?

First of all - lets figure out how did it happen that we fell behind? To do so lets have a look at our GitHub repo (the one we created back here - https://winter-in-hnh.blogspot.ru/2017/01/haven-and-hearth-client.html when we forked Amber repository from romovs):
As you can see above our current branch is 45 commits behind master branch of romovs Amber repository. To see what has changed you can click on "Compare" button, make sure to select your fork as base, and romovs/amber as head, you should see something something like the following:
Here you can see the list of all 45 commits that we've missed so far, and in "Files changed" tab we will be able to see all  files that were changed. One thing to note - is that GitHub currently is not aware of changes we've done - all the changes we were doing so far (https://winter-in-hnh.blogspot.ru/2017/01/extending-haven-and-hearth-client.html and https://winter-in-hnh.blogspot.ru/2017/01/lets-add-buttons.html) were pretty much invisible for GitHub - we were doing all of them locally in our local git environment. So before we get into merging upstream changes, lets commit out own changes.

Commiting changes

To commit changes - right-click your project name in "Project Explorer" select "Team" and then "Commit":
This will bring up "Staging" tab - where you will be presented with changes that you've done so far - select all the files you've changed and write a commit message.
Make sure that you "Commit and Push" your changes - commit will only commit changes to your local repository, while Push, will also get them to remote - GitHub one. Once completed we should see our changes on GitHub:

Unfortunately we can no longer automatically merge, because our changes conflict with some of the changes done to upstream.

How do we merge?

Ok, so we are not at the stage when we would really want to push anything to Amber, so we'll need to change our changes to co-exist with what was modified. There are two GitHub articles that describe what we are going to do next:
So lets follow this guides. First of all lets check what our current local repository is bound to:
Nothing surprising here - this is the repository we created forking romovs/amber and cloned locally to build and modify our Haven and Hearth client (as described in https://winter-in-hnh.blogspot.ru/2017/01/haven-and-hearth-client.html). Now lets add an "upstream" repository:
With "upstream" set we can now fetch content from it:
Let's make sure we are on the "master" branch of our "origin":
Since we expect some merging problems lets use Eclipse IDE merging tool.
What we are planning to do, is to merge our Local repository and 'upstream/master' branch and then commit it to our remote 'origin/master', so first thing to do here is to configure what we will be merging:
So we have conflicts - conflicting places are marked in "Project Explorer" and can be identified in source by special tokens:
So lets start the "Merge tool" and fix our conflicts:
Once complete you should see a "Merged master" message next to your project name in "Project Explorer":
Now it's time to commit our merge the same way we commited our changes in the beginning of the article.

Observing consequencies

Once evertyhing is commited and pushed lets have a look at where we end up. On GitHub we now can compare romovs/amber repository as base to our repository as HEAD:

According to GitHub comparation we are two commits ahead - and that is exactly where we would want to be - ahead of our upstream. Lets try to compile the result.. and it works!

Well done!

If we don't frequently sync to upstreams - we fall behind and the time would come when a change is done to the client that obsoletes our working copy forcing us to re-sync. The further away we are from current state of upstream client - the more is probability that we'll have to merge conflicting changes - so we should be catching up with upstream as frequent as possible to avoid complex merges.

On high level merging is pretty simple:
  1. Make sure you have your upstream repository configured
  2. Fetch changes from your configured upstream repository
  3. Make sure you are on the "master" branch of your forked repository
  4. Merge the changes
  5. If you are lucky - all changes would be a simple fast-forward, if not - resolve conflicts
  6. Rebuild the project
  7. Enjoy