Auto format code in Komodo

This is a macro in Javascript to align, fixing and make code auto-beautified. Tested with Komodo Edit 8/9.

/*
Install dependencies:

  pip install --upgrade autopep8
  pip3 install --upgrade autopep8
  brew install astyle
  brew install tidy-html5
  brew install homebrew/php/php-cs-fixer
  gem install ruby-beautify

*/
komodo.assertMacroVersion(3);
if (komodo.view.scintilla) {
    komodo.view.scintilla.focus();
} // bug 67103
var koDoc = (komodo.koDoc === undefined ? komodo.document : komodo.koDoc);
var formatter;
var language = koDoc.language;
var cannot_tidy_selection = false;

switch (language) {
case 'C#':
    cannot_tidy_selection = true;
    formatter = 'astyle --style=ansi --mode=cs --convert-tabs --indent=spaces=4 %F > /dev/null 2>&1; cat %F';
    break;
case 'C++':
    cannot_tidy_selection = true;
    formatter = 'astyle --style=linux --mode=c --convert-tabs --indent=spaces=4 %F > /dev/null 2>&1; cat %F';
    break;
case 'CSS':
    formatter = 'csstidy - --preserve_css=true --lowercase_s=true --case_properties=true --sort_properties=true --remove_bslash=false --silent=true --template=medium';
    break;
case 'HTML5':
case 'HTML':
    cannot_tidy_selection = true;
    formatter = 'tidy -q -i -b -c -w 120 --show-warnings no --show-errors 0 --tidy-mark no --css-prefix block --drop-proprietary-attributes yes --anchor-as-name no --enclose-text yes';
    break;
case 'Java':
    cannot_tidy_selection = true;
    formatter = 'astyle --style=java --mode=java --convert-tabs --indent=spaces=4 %F > /dev/null 2>&1; cat %F';
    break;
case 'Perl':
    formatter = 'perltidy';
    break;
case 'PHP':
    cannot_tidy_selection = true;
    formatter = 'php php-cs-fixer.phar fix %F';
    break;
case 'Python':
case 'Python3':
    cannot_tidy_selection = true;
    formatter = 'autopep8 --in-place --aggressive %F';
    break;
case 'Ruby':
    formatter = 'rbeautify.rb -';
    break;
case 'XSLT':
    cannot_tidy_selection = true;
    formatter = 'tidy -q -xml -i -w 120 --show-warnings no --show-errors 0 --tidy-mark no';
    break;
case 'XML':
    cannot_tidy_selection = true;
    formatter = 'xmllint --format --recover -';
    break;
default:
    alert("Syntax Undefined, Add Case to Macro " + language);
    return null;
}

// Save cursor Position
var currentPos = komodo.editor.currentPos;
try {
    // Save the file, Check Changes with "File -> Show Unsaved Changes"
    komodo.doCommand('cmd_save');
    // Group operations in a single undo
    // Select Buffer, pipe it into formatter.
    var text_not_selected = cannot_tidy_selection || komodo.editor.selText == "";
    if (text_not_selected) {
        komodo.doCommand('cmd_selectAll');
        ko.run.runEncodedCommand(window, formatter + " {'insertOutput': True}");
    } else {
        ko.run.runEncodedCommand(window, formatter + " {'insertOutput': True, 'operateOnSelection': True}");
    }
    komodo.doCommand('cmd_cleanLineEndings');
    if (text_not_selected) {
        // Reload file
        var view = ko.views.manager.currentView;
        if (view) {
            // Ref: http://docs.activestate.com/komodo/9.0/macroapi.html#macroapi_file
            var currentURI = koDoc.file.URI,
                currentLine = view.scimoz.lineFromPosition(komodo.editor.currentPos);
            // now close and re-open
            view.closeUnconditionally(); // or close()
            ko.open.URI(currentURI + "#" + currentLine);
        }
    }
    StatusBar_AddMessage("Formatted");
}
catch (e) {
    alert(e);
}
finally {
    // End Undo Action to Avoid Edit Buffer Corruption
    // komodo.editor.endUndoAction();
    return true;
}

Go language in weekend

I'm curious about Go language. I found it more like Python than C, such as import, len, multiple assignment...

Very good resources on it:
http://openmymind.net/The-Little-Go-Book/

https://www.gitbook.com/book/codegangsta/building-web-apps-with-go/details

Woodwork resources

Woodworking for beginners
https://www.gutenberg.org/files/43604/43604-h/43604-h.htm

CARPENTRY FOR BOYShttps://www.gutenberg.org/files/20763/20763-h/20763-h.htm

Joints
https://www.gutenberg.org/files/21531/21531-h/21531-h.htm

Handwork in wood
https://www.gutenberg.org/files/20846/20846-h/20846-h.htm

Seasoning of wood
https://www.gutenberg.org/files/26598/26598-h/26598-h.htm

A new setup for web reverse proxy

I'm feed up with setup /etc/hosts and nginx just to have some name proxy to a port on localhost.

I think there is a better way to do it.

My idea is: Browser <-> dnsmasq (dns) / (proxy server) Squid <-> Python / nginx / ...

http://passingcuriosity.com/2013/dnsmasq-dev-osx/

http://derpturkey.com/squid-as-a-reverse-proxy/

Real easte references in Japan

Look up for actual transaction real easte price:
http://www.land.mlit.go.jp/webland/servlet/MainServlet

Glossary
http://www.nomu.com/glossary/law/

Some interesting terms:

おとり広告
is some kind of ads for "luring" customers, with unfeasible ads to trade, or settled false deal. This action is forbid by law.

http://www.nomu.com/glossary/law/417.html

Google Analytics quick reference

Documents:

https://developers.google.com/analytics/devguides/reporting/core/dimsmets

https://developers.google.com/analytics/devguides/reporting/embed/v1/devguide
https://ga-dev-tools.appspot.com/embed-api/basic-dashboard/

Test query
https://ga-dev-tools.appspot.com/query-explorer/
https://developers.google.com/apis-explorer/#p/analytics/v3/analytics.data.ga.get

Embedded GA chart in other website
https://developers.google.com/analytics/devguides/reporting/embed/v1/component-reference#datachart-options

Visualization
https://developers.google.com/chart/

Create a new:
https://ga-dev-tools.appspot.com/embed-api/custom-components/

PHP strftime

It took me hours to debug why a template return null, only when using Japanese locale.
Finally I found out.
strftime often used to format nicely date in Japanese, but it will easily broken in production.

There is a workaround to solve this:

function ensureUtf8($text) {
    return iconv(mb_detect_encoding(
        $text, mb_detect_order(), true), "UTF-8", $text    
    );
}


function _strftime($format, $time = null) {
    return ensureUtf8(strftime($format, $time));
}

New tools

https://github.com/Thibaut/devdocs
I often write code while on commuting train. Local documents are very precious to develop locally without Internet connection.
Days before, I often collect manually by downloading HTML versions (mostly from Readthedocs).
devdocs tool provides much more elegant ways to download and search documents.

Interesting Python style

LBYL

Look before you leap. This coding style explicitly tests for pre-conditions before making calls or lookups. This style contrasts with the EAFPapproach and is characterized by the presence of many if statements.

In a multi-threaded environment, the LBYL approach can risk introducing a race condition between “the looking” and “the leaping”. For example, the code, if key in mapping: return mapping[key] can fail if another thread removes key from mapping after the test, but before the lookup. This issue can be solved with locks or by using the EAFP approach.

EAFP

Easier to ask for forgiveness than permission. This common Python coding style assumes the existence of valid keys or attributes and catches exceptions if the assumption proves false. This clean and fast style is characterized by the presence of many try and except statements. The technique contrasts with the LBYL style common to many other languages such as C.

These are excerpt from https://docs.python.org/3/glossary.html#term-lbyl

Use EAFP is the best for Python and programming pleasure :D

Experience with RiotJS

RiotJS is really nice. Small and do the job!
https://github.com/muut/riotjs

There are some pitfall.

* To mount other tags (defined somewhere outside), I need to put it inside an event:

this.on('mount', function() { riot.mount('other-tag', ...) }

* If you see "onXYZ" on a rendered tag, it meant the event handler wasn't binded correctly. Most of the time, it might because of unresolved handeler.

I don't know when yourHandler is enough, or I must write parent.yourHandler

<ul each={ items}> <li onclick={ parent.yourHandler } ...

* Remember the scope by set self = this first in the script. It's really helpful!

Better Java experience

• IntellJ IDEA! Easier to write with lot of support for Java
• http://projectlombok.org/features/index.html less boilerplate code
• http://velocity.apache.org/engine/devel/user-guide.html for templating
• http://commons.apache.org/ libraries
• https://github.com/google/guava

IntelliJ disable clipboard handling

If you use a clipboard manager like ClipMenu, it will be mal-functional in IntelliJ (Idea, PyCharm, ...)
IntelliJ has it own clipboard handling. To disable it, it's tricky.

http://stackoverflow.com/questions/27949064/how-to-disable-android-studio-intellij-idea-clipboard-management-on-osx

To solve this:

- Open *.app/Contents/bin/idea.properties

- Add a line: ide.mac.useNativeClipboard=True

- Enjoy!

Libraries for analyze Janpanese text

https://github.com/SamuraiT/mecab-python3
http://mecab.googlecode.com/svn/trunk/mecab/doc/index.html


https://github.com/Christoph-D/Japanese-Tools/

Run many services on same server

If a service goes wrong, using 100% -> server is unresponsive, down!

To address this, I finally found:

cgroups: Control CPU/Mem resources for group of process

https://wiki.archlinux.org/index.php/cgroups

nice, cpulimit: Control CPU utilization level.

coreos + docker -> everything will be under measured.

Open big files in Komodo

Although Komodo is great, it will be broken on big files. The reason is costly syntax highlight.

Big files can be: too many lines; or big filesize.

To overcome this, create and run this macro "once".
After running, it will be persisted as a preference.
My setting is 200Kb and 5000 lines limit : )

// Let Komodo able to open big files by disable Syntax Highlight
var gprefs = Components.classes["@activestate.com/koPrefService;1"].
  getService(Components.interfaces.koIPrefService).prefs;
gprefs.setLongPref("documentByteCountThreshold", 200000);
gprefs.setLongPref("documentLineCountThreshold", 5000);
alert('All was set. Openning big files is OK right now');

Ruby notes

Learn Ruby language via style guide is interesting
https://github.com/bbatsov/ruby-style-guide

Goodies:
http://docs.ruby-lang.org/en/2.2.0/Enumerable.html
http://docs.ruby-lang.org/en/2.2.0/syntax/modules_and_classes_rdoc.html

My discoveries:

print : echo bare text

p: print object as it as, with new line

puts: echo text with new line

alias create alias for existing method, var

ensure always executed at block termination

block comment

=begin 

=end

undef remove function definition

convention:

Eg: eql?

method with a question mark return a bool value

Eg: delete! (bang symbol)

method with exclamation ! indicates that it's destructive: change object than a copy

Eg: name=

method ends with = is a setter

block can be wrapped by "do .. end" or { }

run by "yield"

Check for existence via if block_given?

:symbol 

storage optimized string

Global names is from Kernel module

replace string once:

string['search']= 'replace'

replace all

string.gsub 'search', 'replace'

regex

string[/regex/]

regex replace uses the same syntax

string.split(/, /) you might guess

Object methods:

obj.to_s

obj.class

string.to_sym -> :symbol

num_string.to_i -> integer

range.to_a -> array

%w[ string1, ... ]

%i [ symbol1, ...]

[num1, ... ]

array << x # Add x to array (immuttable)

set operator: intersect &, diffeence -, union | 

array.join sep

array.shift

Hash.new(default_value)

hash.key?

hash.value?

hash.keys

hash.values

Gem gotcha

If your local machine's hostname is set to your name, you will likely hit a trouble while installing gem:

"your-dns-needs-immediate-attention"

That's all I know. That message is oversay!

What to do is: set your hostname to a real top-level domain.
If it's a local machine, set it to "your-name.local"
Bold characters must be exactly typed!

# Linux

hostname -s your-name.local 

# Mac OSX

scutil --set HostName your-name.local

Dreamy app stack

If choosing my own stack, I will choose:

- Python: Beauty, simple, powerful. It's slow but productive. Anything has tradeoff ; )
Framework: Tornado

- Cassandra: Autoscale, SQL-like query, Fast write, able to tune consistency: What can I ask more for a database!

- WeedFS: Distribute storage.

Find PHP bottleneck!

There are many ways, such as Xhprof. However, because I already had Xdebug I will use Xdebug to profile the app; and setup Xdebug seems easier than Xhprof.

1. First set these in .htaccess (or php.ini)

php_value xdebug.profiler_enable 1
php_value xdebug.profiler_output_dir "/var/xdebug/"

2. Make sure that /var/xdebug/ is writable by web app, or change it to your folder.

3. Run the request to benchmark.

4. Each request will result in a new file /var/xdebug/cachegrind.out.NNNN

5. To read these stats needs brew install qcachegrind on Mac OSX, or kcachegrind on Linux.
After installing them, run, eg: qcachegrind /var/xdebug/cachegrind.out.NNNN

6. Examine by clicking on "self" column; consider function that take longest time. Well from here we need to think about what caused these bottlenecks.

Custom Komodo look and feel

Komodo's user interface can be "styled" using CSS!
Komodo is written using Mozilla framework. Internally it's HTML/CSS with Javascript/Python API.

Howto:

• Install "DOM Inspector" add-on. Use this to find out element ID, class you want to style, similar as you inspect a website using development tool
• Create  ~/Library/Application Support/KomodoEdit/8.5/XRE/chrome/userChrome.css
• Write your styles ; )

About Cassandra

https://ria101.wordpress.com/2010/02/24/hbase-vs-cassandra-why-we-moved/

- About CAP theorem, Cassandra offers consistent level on reading/writing (selectable CAP tradeoffs)
- Level ALL
- Level QUORUM
- Level ONE
- Symmetric system: no single point of failures. Equal role for all nodes using P2P Gossip
- Simple admin and scale

https://www.igvita.com/2012/02/06/sstable-and-log-structured-storage-leveldb/

SSTable: Sorted String Table

http://www.pythian.com/blog/my-first-experience-with-cassandra-part-1/

Auto reload while development

While developing web application using Python (or Ruby), I often run web server by ourself. If a bug was introduced, web server would die. After debugged it,  I have to re-run it manually! Well, today I figure out a way to restart it automatically ^^. Basically a Bash script will loop and restart on your behalf.

To having this work for you, replace PYTHON_APP (or entire script function) to your specific needs.
Have fun!

#!/bin/bash
SLEEP_INTERVAL=2 # second
PYTHON_APP="python3 -m tasty serve:web --port=9100"
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
script () {
  cd $DIR
  PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1 PYTHONDONTWRITEBYTECODE=1 PYTHONDEBUG=1 PYTHONIOENCODING=utf8 $PYTHON_APP
}
echo "Development mode."
until script; do
    echo "Crashed with exit code $?.  Respawning.." >&2
    sleep $SLEEP_INTERVAL
done

Ruby on Rails reading list

Ruby gotchas:
http://bit.ly/RubyGotchas
http://blog.elpassion.com/Ruby-gotchas/

Prepare for errors:
http://6ftdan.com/allyourdev/2014/12/27/assume-everything-will-break-development/#HJIX

Handle errors:
http://smartinez87.github.io/exception_notification/

Protect by throttle and blocking:
https://github.com/kickstarter/rack-attack

Generate diagram:
https://github.com/voormedia/rails-erd

Better console
http://pryrepl.org/
https://github.com/rweng/pry-rails

Translation tools
https://github.com/glebm/i18n-tasks

Muli-languages models
https://github.com/globalize/globalize