[ACCEPTED]-What is in your Mathematica tool bag?-wolfram-mathematica

Accepted answer
Score: 59

One of the nice things about the Mathematica 51 notebook interface is that it can evaluate 50 expressions in any language, not just Mathematica. As 49 a simple example, consider creating a new 48 Shell input cell type that passes the contained 47 expression to the operating system shell 46 for evaluation.

First, define a function 45 that delegates evaluation of a textual command 44 to the external shell:

shellEvaluate[cmd_, _] := Import["!"~~cmd, "Text"]

The second argument 43 is needed and ignored for reasons that will 42 become apparent later. Next, we want to 41 create a new style called Shell:

  1. Open a new notebook.
  2. Select the menu item Format/Edit Stylesheet...
  3. In the dialog, beside Enter a style name: type Shell.
  4. Select the cell bracket beside the new style.
  5. Select the menu item Cell/Show Expression
  6. Overwrite the cell expression with the Step 6 Text given below.
  7. Once again, select the menu item Cell/Show Expression
  8. Close the dialog.

Use the following 40 cell expression as the Step 6 Text:

Cell[StyleData["Shell"],
 CellFrame->{{0, 0}, {0.5, 0.5}},
 CellMargins->{{66, 4}, {0, 8}},
 Evaluatable->True,
 StripStyleOnPaste->True,
 CellEvaluationFunction->shellEvaluate,
 CellFrameLabels->{{None, "Shell"}, {None, None}},
 Hyphenation->False,
 AutoQuoteCharacters->{},
 PasteAutoQuoteCharacters->{},
 LanguageCategory->"Formula",
 ScriptLevel->1,
 MenuSortingValue->1800,
 FontFamily->"Courier"]

Most of this expression 39 was copied directly form the built-in Program style. The 38 key changes are these lines:

 Evaluatable->True,
 CellEvaluationFunction->shellEvaluate,
 CellFrameLabels->{{None, "Shell"}, {None, None}},

Evaluatable enables the 37 SHIFT+ENTER functionality for the cell. Evaluation 36 will call the CellEvaluationFunction passing the cell content 35 and content type as arguments (shellEvaluate ignores 34 the latter argument). CellFrameLabels is just a nicety 33 that let's the user identify that this cell 32 is unusual.

With all of this in place, we 31 can now enter and evaluate a shell expression:

  1. In the notebook created in the steps above, create an empty cell and select the cell bracket.
  2. Select the menu item Format/Style/Shell.
  3. Type a valid operating system shell command into the cell (e.g. 'ls' on Unix or 'dir' on Windows).
  4. Press SHIFT+ENTER.

It 30 is best to keep this defined style in a 29 centrally located stylesheet. Furthermore, evaluation 28 functions like shellEvaluate are best defined as stubs 27 using DeclarePackage in init.m. The details of both of these 26 activities are beyond the scope of this 25 response.

With this functionality, one can 24 create notebooks that contain input expressions 23 in any syntax of interest. The evaluation 22 function can be written in pure Mathematica, or 21 delegate any or all parts of the evaluation 20 to an external agency. Be aware that there 19 are other hooks that relate to cell evaluation, like 18 CellEpilog, CellProlog and CellDynamicExpression.

A common pattern involves writing 17 the input expression text to a temporary 16 file, compiling the file in some language, running 15 the program and capturing the output for 14 ultimate display in the output cell. There 13 are plenty of details to address when implementing 12 a full solution of this kind (like capturing 11 error messages properly), but one must appreciate 10 the fact that it is not only possible to 9 do things like this, but practical.

On a 8 personal note, it is features like this 7 that makes the notebook interface the center 6 of my programming universe.

Update

The following 5 helper function is useful for creating such 4 cells:

evaluatableCell[label_String, evaluationFunction_] :=
  ( CellPrint[
      TextCell[
        ""
      , "Program"
      , Evaluatable -> True
      , CellEvaluationFunction -> (evaluationFunction[#]&)
      , CellFrameLabels -> {{None, label}, {None, None}}
      , CellGroupingRules -> "InputGrouping"
      ]
    ]
  ; SelectionMove[EvaluationNotebook[], All, EvaluationCell]
  ; NotebookDelete[]
  ; SelectionMove[EvaluationNotebook[], Next, CellContents]
  )

It is used thus:

shellCell[] := evaluatableCell["shell", Import["!"~~#, "Text"] &]

Now, if shellCell[] is evaluated, the 3 input cell will be deleted and replaced 2 with a new input cell that evaluates its 1 contents as a shell command.

Score: 40

Todd Gayley (Wolfram Research) just send 19 me a nice hack which allows to "wrap" built-in 18 functions with arbitrary code. I feel that 17 I have to share this useful instrument. The 16 following is Todd's answer on my question.

A bit 15 of interesting (?) history: That style of 14 hack for "wrapping" a built-in 13 function was invented around 1994 by Robby 12 Villegas and I, ironically for the function 11 Message, in a package called ErrorHelp that 10 I wrote for the Mathematica Journal back 9 then. It has been used many times, by 8 many people, since then. It's a bit of an 7 insider's trick, but I think it's fair 6 to say that it has become the canonical 5 way of injecting your own code into the 4 definition of a built-in function. It 3 gets the job done nicely. You can, of course, put 2 the $inMsg variable into any private 1 context you wish.

Unprotect[Message];

Message[args___] := Block[{$inMsg = True, result},
   "some code here";
   result = Message[args];
   "some code here";
   result] /; ! TrueQ[$inMsg]

Protect[Message];
Score: 30

I've mentioned this before, but the tool I 71 find most useful is an application of Reap and 70 Sow which mimics/extends the behavior of GatherBy:

SelectEquivalents[x_List,f_:Identity, g_:Identity, h_:(#2&)]:=
   Reap[Sow[g[#],{f[#]}]&/@x, _, h][[2]];

This 69 allows me to group lists by any criteria 68 and transform them in the process. The 67 way it works is that a criteria function 66 (f) tags each item in the list, each item 65 is then transformed by a second supplied 64 function (g), and the specific output is 63 controlled by a third function (h). The 62 function h accepts two arguments: a tag 61 and a list of the collected items that have 60 that tag. The items retain their original 59 order, so if you set h = #1& then you get an unsorted 58 Union, like in the examples for Reap. But, it can be used 57 for secondary processing.

As an example of 56 its utility, I've been working with Wannier90 which 55 outputs the spatially dependent Hamiltonian 54 into a file where each line is a different 53 element in the matrix, as follows

rx ry rz i j Re[Hij] Im[Hij]

To turn 52 that list into a set of matrices, I gathered 51 up all sublists that contain the same coordinate, turned 50 the element information into a rule (i.e. {i,j}-> Re[Hij]+I 49 Im[Hij]), and then turned the collected 48 rules into a SparseArray all with the one liner:

SelectEquivalents[hamlst, 
      #[[;; 3]] &, 
      #[[{4, 5}]] -> (Complex @@ #[[6 ;;]]) &, 
      {#1, SparseArray[#2]} &]

Honestly, this 47 is my Swiss Army Knife, and it makes complex 46 things very simple. Most of my other tools 45 are somewhat domain specific, so I'll probably 44 not post them. However, most, if not all, of 43 them reference SelectEquivalents.

Edit: it doesn't completely 42 mimic GatherBy in that it cannot group multiple 41 levels of the expression as simply as GatherBy can. However, Map works 40 just fine for most of what I need.

Example: @Yaroslav 39 Bulatov has asked for a self-contained example. Here's 38 one from my research that has been greatly 37 simplified. So, let's say we have a set 36 of points in a plane

In[1] := pts = {{-1, -1, 0}, {-1, 0, 0}, {-1, 1, 0}, {0, -1, 0}, {0, 0, 0}, 
 {0, 1, 0}, {1, -1, 0}, {1, 0, 0}, {1, 1, 0}}

and we'd like to reduce 35 the number of points by a set of symmetry 34 operations. (For the curious, we are generating 33 the little group of each point.) For this example, let's 32 use a four fold rotation axis about the 31 z-axis

In[2] := rots = RotationTransform[#, {0, 0, 1}] & /@ (Pi/2 Range[0, 3]);

Using SelectEquivalents we can group the points that 30 produce the same set of images under these 29 operations, i.e. they're equivalent, using 28 the following

In[3] := SelectEquivalents[ pts, Union[Through[rots[#] ] ]& ] (*<-- Note Union*)
Out[3]:= {{{-1, -1, 0}, {-1, 1, 0}, {1, -1, 0}, {1, 1, 0}},
          {{-1, 0, 0}, {0, -1, 0}, {0, 1, 0}, {1, 0, 0}},
          {{0,0,0}}}

which produces 3 sublists containing 27 the equivalent points. (Note, Union is absolutely 26 vital here as it ensures that the same image 25 is produced by each point. Originally, I 24 used Sort, but if a point lies on a symmetry 23 axis, it is invariant under the rotation 22 about that axis giving an extra image of 21 itself. So, Union eliminates these extra images. Also, GatherBy would 20 produce the same result.) In this case, the 19 points are already in a form that I will 18 use, but I only need a representative point 17 from each grouping and I'd like a count 16 of the equivalent points. Since, I don't 15 need to transform each point, I use the 14 Identity function in the second position. For the 13 third function, we need to be careful. The 12 first argument passed to it will be the 11 images of the points under the rotations 10 which for the point {0,0,0} is a list of four identical 9 elements, and using it would throw off the 8 count. However, the second argument is 7 just a list of all the elements that have 6 that tag, so it will only contain {0,0,0}. In 5 code,

In[4] := SelectEquivalents[pts,  
             Union[Through[rots[#]]]&, #&, {#2[[1]], Length[#2]}& ]
Out[4]:= {{{-1, -1, 0}, 4}, {{-1, 0, 0}, 4}, {{0, 0, 0}, 1}}

Note, this last step can just as easily 4 be accomplished by

In[5] := {#[[1]], Length[#]}& /@ Out[3]

But, it is easy with 3 this and the less complete example above 2 to see how very complex transformations 1 are possible with a minimum of code.

Score: 25

This is not a complete resource, so I'm 8 throwing it here in the answers section, but 7 I have found it very useful when figuring 6 out speed issues (which, unfortunately, is 5 a large part of what Mathematica programming 4 is about).

timeAvg[func_] := Module[
{x = 0, y = 0, timeLimit = 0.1, p, q, iterTimes = Power[10, Range[0, 10]]},
Catch[
 If[(x = First[Timing[(y++; Do[func, {#}]);]]) > timeLimit,
    Throw[{x, y}]
    ] & /@ iterTimes
 ] /. {p_, q_} :> p/iterTimes[[q]]
];
Attributes[timeAvg] = {HoldAll};

Usage is then simply timeAvg@funcYouWantToTest.

EDIT: Mr. Wizard 3 has provided a simpler version that does 2 away with Throw and Catch and is a bit easier to parse:

SetAttributes[timeAvg, HoldFirst]
timeAvg[func_] := Do[If[# > 0.3, Return[#/5^i]] & @@ 
                     Timing @ Do[func, {5^i}]
                     ,{i, 0, 15}]

EDIT: Here's 1 a version from acl (taken from here):

timeIt::usage = "timeIt[expr] gives the time taken to execute expr, \
  repeating as many times as necessary to achieve a total time of 1s";

SetAttributes[timeIt, HoldAll]
timeIt[expr_] := Module[{t = Timing[expr;][[1]], tries = 1},
  While[t < 1., tries *= 2; t = Timing[Do[expr, {tries}];][[1]];]; 
  t/tries]
Score: 21

Internal`InheritedBlock

I have learned recently the existence of 18 such useful function as Internal`InheritedBlock, from this message of Daniel Lichtblau in the official 17 newsgroup.

As I understand, Internal`InheritedBlock allows to pass 16 a copy of an outbound function inside the 15 Block scope:

In[1]:= Internal`InheritedBlock[{Message},
Print[Attributes[Message]];
Unprotect[Message];
Message[x___]:=Print[{{x},Stack[]}];
Sin[1,1]
]
Sin[1,1]
During evaluation of In[1]:= {HoldFirst,Protected}
During evaluation of In[1]:= {{Sin::argx,Sin,2},{Internal`InheritedBlock,CompoundExpression,Sin,Print,List}}
Out[1]= Sin[1,1]
During evaluation of In[1]:= Sin::argx: Sin called with 2 arguments; 1 argument is expected. >>
Out[2]= Sin[1,1]

I think this function can be very 14 useful for everyone who need to modify built-in 13 functions temporarily!

Comparison with Block

Let us define some 12 function:

a := Print[b]

Now we wish to pass a copy of this 11 function into the Block scope. The naive trial 10 does not give what we want:

In[2]:= Block[{a = a}, OwnValues[a]]

During evaluation of In[9]:= b

Out[2]= {HoldPattern[a] :> Null}

Now trying to 9 use delayed definition in the first argument 8 of Block (it is an undocumented feature too):

In[3]:= Block[{a := a}, OwnValues[a]]
Block[{a := a}, a]

Out[3]= {HoldPattern[a] :> a}

During evaluation of In[3]:= b

We 7 see that in this case a works but we have 6 not got a copy of the original a inside of 5 the Block scope.

Now let us try Internal`InheritedBlock:

In[5]:= Internal`InheritedBlock[{a}, OwnValues[a]]

Out[5]= {HoldPattern[a] :> Print[b]}

We have got a 4 copy of the original definition for a inside 3 of the Block scope and we may modify it in the 2 way we want without affecting the global 1 definition for a!

Score: 20

Mathematica is a sharp tool, but it can 34 cut you with its somewhat untyped behaviour and avalanches of cryptic 33 diagnostic messages. One way to deal with this is to define 32 functions following this idiom:

ClearAll@zot
SetAttributes[zot, ...]
zot[a_] := ...
zot[b_ /; ...] := ...
zot[___] := (Message[zot::invalidArguments]; Abort[])

That is a 31 lot of boilerplate, which I'm frequently 30 tempted to skip. Especially when prototyping, which 29 happens a lot in Mathematica. So, I use 28 a macro called define that allows me to stay disciplined, with 27 much less boilerplate.

A basic usage of define is 26 like this:

define[
  fact[0] = 1
; fact[n_ /; n > 0] := n * fact[n-1]
]

fact[5]

120

It doesn't look like much at first, but 25 there are some hidden benefits. The first 24 service that define provides is that it automatically 23 applies ClearAll to the symbol being defined. This 22 ensures that there are no leftover definitions 21 -- a common occurrence during the initial 20 development of a function.

The second service 19 is that the function being defined is automatically 18 "closed". By this I mean that 17 the function will issue a message and abort 16 if it is invoked with an argument list that 15 is not matched by one of the definitions:

fact[-1]

define::badargs: There is no definition for 'fact' applicable to fact[-1].
$Aborted

This 14 is the primary value of define, which catches 13 a very common class of error.

Another convenience 12 is a concise way to specify attributes on 11 the function being defined. Let's make 10 the function Listable:

define[
  fact[0] = 1
; fact[n_ /; n > 0] := n * fact[n-1]
, Listable
]

fact[{3, 5, 8}]

{6, 120, 40320}

In addition to all of the 9 normal attributes, define accepts an additional 8 attribute called Open. This prevents define from 7 adding the catch-all error definition to 6 the function:

define[
  successor[x_ /; x > 0] := x + 1
, Open
]

successor /@ {1, "hi"}

{2, successor["hi"]}

Multiple attributes may be 5 defined for a function:

define[
  flatHold[x___] := Hold[x]
, {Flat, HoldAll}
]

flatHold[flatHold[1+1, flatHold[2+3]], 4+5]

Hold[1 + 1, 2 + 3, 4 + 5]

Without further ado, here 4 is the definition of define:

ClearAll@define
SetAttributes[define, HoldAll]
define[body_, attribute_Symbol] := define[body, {attribute}]
define[body:(_Set|_SetDelayed), attributes_List:{}] := define[CompoundExpression[body], attributes]
define[body:CompoundExpression[((Set|SetDelayed)[name_Symbol[___], _])..], attributes_List:{}] :=
  ( ClearAll@name
  ; SetAttributes[name, DeleteCases[attributes, Open]]
  ; If[!MemberQ[attributes, Open]
    , def:name[___] := (Message[define::badargs, name, Defer@def]; Abort[])
    ]
  ; body
  ;
  )
def:define[___] := (Message[define::malformed, Defer@def]; Abort[])

define::badargs = "There is no definition for '``' applicable to ``.";
define::malformed = "Malformed definition: ``";

The exhibited implementation 3 supports neither up-values nor currying, nor 2 patterns more general than simple function 1 definition. It remains useful, however.

Score: 16

Start without a blank notebook open

I was bothered by having Mathematica start 21 with a blank notebook open. I could close 20 this notebook with a script, but it would 19 still flash open briefly. My hack is to 18 create a file Invisible.nb containing:

Notebook[{},Visible->False]

And add this to 17 my Kernel\init.m :

If[Length[Notebooks["Invisible*"]] > 0, 
  NotebookClose[Notebooks["Invisible*"][[1]]]
]

SetOptions[$FrontEnd,
  Options[$FrontEnd, NotebooksMenu] /. 
    HoldPattern["Invisible.nb" -> {__}] :> Sequence[]
]

I now start Mathematica by opening 16 Invisible.nb

There may be a better way, but this has 15 served me well.


Customized Fold and FoldList

Fold[f, x] is made equivalent to Fold[f, First@x, Rest@x]

Incidentally, I believe this may find its way into a future version of Mathematica.

Surprise! This has been implemented, though it is presently undocumented. I 14 am informed that it was implemented in 2011 13 by Oliver Ruebenkoenig, apparently not long 12 after I posted this. Thank you Oliver Ruebenkoenig!

Unprotect[Fold, FoldList]

Fold[f_, h_[a_, b__]] := Fold[f, Unevaluated @ a, h @ b]
FoldList[f_, h_[a_, b__]] := FoldList[f, Unevaluated @ a, h @ b]

(* Faysal's recommendation to modify SyntaxInformation *)
SyntaxInformation[Fold]     = {"ArgumentsPattern" -> {_, _, _.}};
SyntaxInformation[FoldList] = {"ArgumentsPattern" -> {_, _., {__}}};

Protect[Fold, FoldList]

Updated 11 to allow this:

SetAttributes[f, HoldAll]
Fold[f, Hold[1 + 1, 2/2, 3^3]]
f[f[1 + 1, 2/2], 3^3]

"Dynamic Partition"

See Mathematica.SE post #7512 for a new version of this function.

Frequently I want to partition 10 a list according to a sequence of lengths.

pseudo-code 9 example:

partition[{1,2,3,4,5,6}, {2,3,1}]

Output: {{1,2}, {3,4,5}, {6}}

I came up with this:

dynP[l_, p_] := 
 MapThread[l[[# ;; #2]] &, {{0} ~Join~ Most@# + 1, #} &@Accumulate@p]

Which 8 I then completed with this, including argument 7 testing:

dynamicPartition[l_List, p : {_Integer?NonNegative ..}] :=
  dynP[l, p] /; Length@l >= Tr@p

dynamicPartition[l_List, p : {_Integer?NonNegative ..}, All] :=
  dynP[l, p] ~Append~ Drop[l, Tr@p] /; Length@l >= Tr@p

dynamicPartition[l_List, p : {_Integer?NonNegative ..}, n__ | {n__}] :=
  dynP[l, p] ~Join~ Partition[l ~Drop~ Tr@p, n] /; Length@l >= Tr@p

The third argument controls what 6 happens to elements beyond the split specification.


Szabolcs's Mathematica tricks

The 5 one I use most frequently is the Paste Tabular 4 Data Palette

CreatePalette@
 Column@{Button["TSV", 
    Module[{data, strip}, 
     data = NotebookGet[ClipboardNotebook[]][[1, 1, 1]];
     strip[s_String] := 
      StringReplace[s, RegularExpression["^\\s*(.*?)\\s*$"] -> "$1"];
     strip[e_] := e;
     If[Head[data] === String, 
      NotebookWrite[InputNotebook[], 
       ToBoxes@Map[strip, ImportString[data, "TSV"], {2}]]]]], 
   Button["CSV", 
    Module[{data, strip}, 
     data = NotebookGet[ClipboardNotebook[]][[1, 1, 1]];
     strip[s_String] := 
      StringReplace[s, RegularExpression["^\\s*(.*?)\\s*$"] -> "$1"];
     strip[e_] := e;
     If[Head[data] === String, 
      NotebookWrite[InputNotebook[], 
       ToBoxes@Map[strip, ImportString[data, "CSV"], {2}]]]]], 
   Button["Table", 
    Module[{data}, data = NotebookGet[ClipboardNotebook[]][[1, 1, 1]];
     If[Head[data] === String, 
      NotebookWrite[InputNotebook[], 
       ToBoxes@ImportString[data, "Table"]]]]]}

Modify external data from within Compile

Recently Daniel Lichtblau showed 3 this method I had never seen before. In 2 my opinion it significantly extends the 1 utility of Compile

ll = {2., 3., 4.};
c = Compile[{{x}, {y}}, ll[[1]] = x; y];

c[4.5, 5.6]

ll

(* Out[1] = 5.6  *)

(* Out[2] = {4.5, 3., 4.}  *)
Score: 16

General PDF/EMF export problems and solutions

1) It is completely unexpected and undocumented, but 45 Mathematica exports and saves graphics in PDF and EPS 44 formats using a set of style definitions 43 that differs from the one used for displaying 42 Notebooks on screen. By default Notebooks 41 are displayed on screen in the "Working" style 40 environment (which is default value for 39 the ScreenStyleEvironment global $FrontEnd option) but are printed in 38 the "Printout" style environment (which is default 37 value for the PrintingStyleEnvironment global $FrontEnd option). When one 36 exports graphics in raster formats such 35 as GIF and PNG or in EMF format Mathematica generates 34 graphics that looks exactly like it looks 33 inside Notebook. It seems that the "Working" style 32 environment is used for rendering in this 31 case. But it is not the case when you export/save 30 anything in PDF or EPS formats! In this 29 case the "Printout" style environment is used by default that differs very deeply from 28 the "Working" style environment. First 27 of all, the "Printout" style environment sets Magnification to 80%. Secondly, it uses its own values 26 for the font sizes of different styles and 25 this results in inconsistent font size changes 24 in the genarated PDF file as compared with 23 the original on-screen representation. The 22 latter can be called FontSize fluctuations which are very annoying. But 21 happily this can be avoided by setting the PrintingStyleEnvironment global $FrontEnd option to "Working":

SetOptions[$FrontEnd, PrintingStyleEnvironment -> "Working"]

2) The common 20 problem with exporting to EMF format is 19 that most of programs (not only Mathematica) generate 18 a file that looks nice at the default size 17 but becomes ugly when you zoom it in. It 16 is because metafiles are sampled at screen resolution fidelity. The quality of the generated 15 EMF file can be enhanced by Magnifying the original 14 graphical object so that exactness of sampling 13 of the original graphics becomes much more 12 precise. Compare two files:

graphics1 = 
  First@ImportString[
    ExportString[Style["a", FontFamily -> "Times"], "PDF"], "PDF"];
graphics2 = Magnify[graphics1, 10];
Export["C:\\test1.emf", graphics1]
Export["C:\\test2.emf", graphics2]

If you insert 11 these files into Microsoft Word and zoom 10 them in you will see that the first "a" has 9 sawtooth on it while the second has not 8 (tested with Mathematica 6).

Another way through ImageResolution was 7 suggested by Chris Degnen (this option has effect at 6 least starting from Mathematica 8):

Export["C:\\test1.emf", graphics1]
Export["C:\\test2.emf", graphics1, ImageResolution -> 300]

3) In Mathematica we have three 5 ways to convert graphics into metafile: via 4 Export to "EMF" (strongly recommended way: produces 3 metafile with highest possible quality), via 2 Save selection As... menu item (produces much lesser precise figure, not recommended) and via Edit ► Copy As ► Metafile menu 1 item (I strongly recommend against this route).

Score: 13

By popular demand, the code to generate 2 the top-10 SO answerers plot (except annotations) using 1 the SO API.

enter image description here

getRepChanges[userID_Integer] :=
 Module[{totalChanges},
  totalChanges = 
   "total" /. 
    Import["http://api.stackoverflow.com/1.1/users/" <> 
      ToString[userID] <> "/reputation?fromdate=0&pagesize=10&page=1",
      "JSON"];
  Join @@ Table[
    "rep_changes" /. 
     Import["http://api.stackoverflow.com/1.1/users/" <> 
       ToString[userID] <> 
       "/reputation?fromdate=0&pagesize=10&page=" <> ToString[page], 
      "JSON"],
    {page, 1, Ceiling[totalChanges/10]}
    ]
  ]

topAnswerers = ({"display_name", 
      "user_id"} /. #) & /@ ("user" /. ("top_users" /. 
      Import["http://api.stackoverflow.com/1.1/tags/mathematica/top-\
answerers/all-time", "JSON"]))

repChangesTopUsers =
  Monitor[Table[
    repChange = 
     ReleaseHold[(Hold[{DateList[
              "on_date" + AbsoluteTime["January 1, 1970"]], 
             "positive_rep" - "negative_rep"}] /. #) & /@ 
        getRepChanges[userID]] // Sort;
    accRepChange = {repChange[[All, 1]], 
       Accumulate[repChange[[All, 2]]]}\[Transpose],
    {userID, topAnswerers[[All, 2]]}
    ], userID];

pl = DateListLogPlot[
  Tooltip @@@ 
   Take[({repChangesTopUsers, topAnswerers[[All, 1]]}\[Transpose]), 
    10], Joined -> True, Mesh -> None, ImageSize -> 1000, 
  PlotRange -> {All, {10, All}}, 
  BaseStyle -> {FontFamily -> "Arial-Bold", FontSize -> 16}, 
  DateTicksFormat -> {"MonthNameShort", " ", "Year"}, 
  GridLines -> {True, None}, 
  FrameLabel -> (Style[#, FontSize -> 18] & /@ {"Date", "Reputation", 
      "Top-10 answerers", ""})]
Score: 13

Caching expressions

I find these functions very helpful to cache 78 any expression. The interesting thing here 77 for these two functions is that the held 76 expression itself is used as a key of the 75 hashtable/symbol Cache or CacheIndex, compared 74 to the well-known memoization in mathematica 73 where you can only cache result if the function 72 is defined like f[x_] := f[x] = ... So you 71 can cache any part of a code, this is useful 70 if a function is to be called several times 69 but just some parts of the code must not 68 be recomputed.

To cache an expression independently 67 of its arguments.

SetAttributes[Cache, HoldFirst];
c:Cache[expr_] := c = expr;

Ex: Cache[Pause[5]; 6]
Cache[Pause[5]; 6]

The second time the expression 66 returns 6 without waiting.

To cache an expression 65 using an alias expression that can depend 64 on an argument of the cached expression.

SetAttributes[CacheIndex, HoldRest];
c:CacheIndex[index_,expr_] := c = expr;

Ex: CacheIndex[{"f",2},x=2;y=4;x+y]

If 63 expr takes some time to compute, it is much 62 faster to evaluate {"f",2} for 61 example to retrieve the cached result.

For 60 a variation of these functions in order 59 to have a localized cache (ie. the cache 58 memory is automatically released outside 57 the Block construct) see this post Avoid repeated calls to Interpolation

Deleting cached values

To delete 56 cached values when you don't know the number 55 of definitions of a function. I consider 54 that definitions have a Blank somewhere 53 in their arguments.

DeleteCachedValues[f_] := 
       DownValues[f] = Select[DownValues[f], !FreeQ[Hold@#,Pattern]&];

To delete cached values 52 when you know the number of definitions 51 of a function (goes slightly faster).

DeleteCachedValues[f_,nrules_] := 
       DownValues[f] = Extract[DownValues[f], List /@ Range[-nrules, -1]];

This 50 uses the fact that definitions of a function 49 are at the end of their DownValues list, cached 48 values are before.

Using symbols to store data and object-like functions

Also here are interesting 47 functions to use symbols like objects.

It 46 is already well known that you can store 45 data in symbols and quickly access them 44 using DownValues

mysymbol["property"]=2;

You can access the list 43 of keys (or properties) of a symbol using 42 these functions based on what dreeves submitted 41 in a post on this site:

SetAttributes[RemoveHead, {HoldAll}];
RemoveHead[h_[args___]] := {args};
NKeys[symbol_] := RemoveHead @@@ DownValues[symbol(*,Sort->False*)][[All,1]];
Keys[symbol_] := NKeys[symbol] /. {x_} :> x;

I use this function 40 a lot to display all infos contained in 39 the DownValues of a symbol:

PrintSymbol[symbol_] :=
  Module[{symbolKeys},
    symbolKeys = Keys[symbol];
    TableForm@Transpose[{symbolKeys, symbol /@ symbolKeys}]
  ];

Finally here 38 is a simple way to create a symbol that 37 behaves like an object in object oriented 36 programming (it just reproduces the most 35 basic behaviour of OOP but I find the syntax 34 elegant) :

Options[NewObject]={y->2};
NewObject[OptionsPattern[]]:=
  Module[{newObject},
    newObject["y"]=OptionValue[y];

    function[newObject,x_] ^:= newObject["y"]+x;
    newObject /: newObject.function2[x_] := 2 newObject["y"]+x;

    newObject
  ];

Properties are stored as DownValues 33 and methods as delayed Upvalues in the symbol 32 created by Module that is returned. I found 31 the syntax for function2 that is the usual 30 OO-syntax for functions in Tree data structure in Mathematica.

For a list 29 of existing types of values each symbol 28 has, see http://reference.wolfram.com/mathematica/tutorial/PatternsAndTransformationRules.html and http://www.verbeia.com/mathematica/tips/HTMLLinks/Tricks_Misc_4.html.

For example try this

x = NewObject[y -> 3];
function[x, 4]
x.function2[5]

You can 27 go further if you want to emulate object 26 inheritance using a package called InheritRules 25 available here http://library.wolfram.com/infocenter/MathSource/671/

You could also store the 24 function definition not in newObject but 23 in a type symbol, so if NewObject returned 22 type[newObject] instead of newObject you 21 could define function and function2 like 20 this outside of NewObject (and not inside) and 19 have the same usage as before.

function[type[object_], x_] ^:= object["y"] + x;
type /: type[object_].function2[x_] := 2 object["y"]+x;

Use UpValues[type] to 18 see that function and function2 are defined 17 in the type symbol.

Further ideas about this 16 last syntax are introduced here https://mathematica.stackexchange.com/a/999/66.

Improved version of SelectEquivalents

@rcollyer: Many 15 thanks for bringing SelectEquivalents to 14 the surface, it's an amazing function. Here 13 is an improved version of SelectEquivalents 12 listed above with more possibilities and 11 using options, this makes it easier to use.

Options[SelectEquivalents] = 
   {
      TagElement->Identity,
      TransformElement->Identity,
      TransformResults->(#2&) (*#1=tag,#2 list of elements corresponding to tag*),
      MapLevel->1,
      TagPattern->_,
      FinalFunction->Identity
   };

SelectEquivalents[x_List,OptionsPattern[]] := 
   With[
      {
         tagElement=OptionValue@TagElement,
         transformElement=OptionValue@TransformElement,
         transformResults=OptionValue@TransformResults,
         mapLevel=OptionValue@MapLevel,
         tagPattern=OptionValue@TagPattern,
         finalFunction=OptionValue@FinalFunction
      }
      ,
      finalFunction[
         Reap[
            Map[
               Sow[
                  transformElement@#
                  ,
                  {tagElement@#}
               ]&
               , 
               x
               , 
               {mapLevel}
            ] 
            , 
            tagPattern
            , 
            transformResults
         ][[2]]
      ]
   ];

Here 10 are examples of how this version can be 9 used:

Using Mathematica Gather/Collect properly

How would you do a PivotTable function in Mathematica?

Mathematica fast 2D binning algorithm

Internal`Bag

Daniel Lichtblau describes here an 8 interesting internal data structure for 7 growing lists.

Implementing a Quadtree in Mathematica

Debugging functions

These two posts point to useful 6 functions for debugging:

How to debug when writting small or big codes using Mathematica? workbench? mma debugger? or something else? (ShowIt)

https://stackoverflow.com/questions/5459735/the-clearest-way-to-represent-mathematicas-evaluation-sequence/5527117#5527117 (TraceView)

Here's 5 another function based on Reap and Sow to 4 extract expressions from different parts 3 of a program and store them in a symbol.

SetAttributes[ReapTags,HoldFirst];
ReapTags[expr_]:=
   Module[{elements},
      Reap[expr,_,(elements[#1]=#2/.{x_}:>x)&];
      elements
   ];

Here's 2 an example

ftest[]:=((*some code*)Sow[1,"x"];(*some code*)Sow[2,"x"];(*some code*)Sow[3,"y"]);
s=ReapTags[ftest[]];
Keys[s]
s["x"]
PrintSymbol[s] (*Keys and PrintSymbol are defined above*)

Other resources

Here's a list of interesting links 1 for learning purpose:

A collection of Mathematica learning resources

Updated here: https://mathematica.stackexchange.com/a/259/66

Score: 12

My utility functions (I have these built 1 in to MASH, which is mentioned in the question):

pr = WriteString["stdout", ##]&;            (* More                           *)
prn = pr[##, "\n"]&;                        (*  convenient                    *)
perr = WriteString["stderr", ##]&;          (*   print                        *)
perrn = perr[##, "\n"]&;                    (*    statements.                 *)
re = RegularExpression;                     (* I wish mathematica             *)
eval = ToExpression[cat[##]]&;              (*  weren't so damn               *)
EOF = EndOfFile;                            (*   verbose!                     *)
read[] := InputString[""];                  (* Grab a line from stdin.        *)
doList[f_, test_] :=                        (* Accumulate list of what f[]    *)
  Most@NestWhileList[f[]&, f[], test];      (*  returns while test is true.   *)
readList[] := doList[read, #=!=EOF&];       (* Slurp list'o'lines from stdin. *)
cat = StringJoin@@(ToString/@{##})&;        (* Like sprintf/strout in C/C++.  *)
system = Run@cat@##&;                       (* System call.                   *)
backtick = Import[cat["!", ##], "Text"]&;   (* System call; returns stdout.   *)
slurp = Import[#, "Text"]&;                 (* Fetch contents of file as str. *)
                                            (* ABOVE: mma-scripting related.  *)
keys[f_, i_:1] :=                           (* BELOW: general utilities.      *)
  DownValues[f, Sort->False][[All,1,1,i]];  (* Keys of a hash/dictionary.     *)
SetAttributes[each, HoldAll];               (* each[pattern, list, body]      *)
each[pat_, lst_, bod_] := ReleaseHold[      (*  converts pattern to body for  *)
  Hold[Cases[Evaluate@lst, pat:>bod];]];    (*   each element of list.        *)
some[f_, l_List] := True ===                (* Whether f applied to some      *)
  Scan[If[f[#], Return[True]]&, l];         (*  element of list is True.      *)
every[f_, l_List] := Null ===               (* Similarly, And @@ f/@l         *)
  Scan[If[!f[#], Return[False]]&, l];       (*  (but with lazy evaluation).   *)
Score: 11

One trick I've used, which allows you to 29 emulate the way most built-in functions 28 work with bad arguments (by sending a message 27 and then returning the whole form unevaluated) exploits 26 a quirk of the way Condition works when used in a 25 defintion. If foo should only work with one 24 argument:

foo[x_] := x + 1;
expr : foo[___] /; (Message[foo::argx, foo, Length@Unevaluated[expr], 1]; 
                    False) := Null; (* never reached *)

If you have more complex needs, it's 23 easy to factor out the argument validation 22 and message generation as an independent 21 function. You can do more elaborate things 20 by using side effects in Condition beyond just generating 19 messages, but in my opinion most of them 18 fall into the "sleazy hack" category 17 and should be avoided if possible.

Also, in 16 the "metaprogramming" category, if 15 you have a Mathematica package (.m) file, you 14 can use the "HeldExpressions" element to get all the expressions 13 in the file wrapped in HoldComplete. This makes tracking 12 things down much easier than using text-based 11 searches. Unfortunately, there's no easy 10 way to do the same thing with a notebook, but 9 you can get all the input expressions using 8 something like the following:

inputExpressionsFromNotebookFile[nb_String] :=
 Cases[Get[nb],
  Cell[BoxData[boxes_], "Input", ___] :>
   MakeExpression[StripBoxes[boxes], StandardForm],
  Infinity]

Lastly, you 7 can use the fact that Module emulates lexical 6 closures to create the equivalent of reference 5 types. Here's a simple stack (which uses 4 a variation the Condition trick for error handling 3 as a bonus):

ClearAll[MakeStack, StackInstance, EmptyQ, Pop, Push, Peek]
 With[{emptyStack = Unique["empty"]},
  Attributes[StackInstance] = HoldFirst;
  MakeStack[] :=
   Module[{backing = emptyStack},
    StackInstance[backing]];

  StackInstance::empty = "stack is empty";

  EmptyQ[StackInstance[backing_]] := (backing === emptyStack);

  HoldPattern[
    Pop[instance : StackInstance[backing_]]] /;
    ! EmptyQ[instance] || (Message[StackInstance::empty]; False) :=
   (backing = Last@backing; instance);

  HoldPattern[Push[instance : StackInstance[backing_], new_]] :=
   (backing = {new, backing}; instance);

  HoldPattern[Peek[instance : StackInstance[backing_]]] /;
    ! EmptyQ[instance] || (Message[StackInstance::empty]; False) :=
   First@backing]

Now you can print the elements 2 of a list in reverse order in a needlessly 1 convoluted way!

With[{stack = MakeStack[], list},
 Do[Push[stack, elt], {elt, list}];

 While[!EmptyQ[stack],
  Print[Peek@stack];
  Pop@stack]]
Score: 11

Printing system symbol definitions without context prepended

The contextFreeDefinition[] function below will attempt to print 26 the definition of a symbol without the most 25 common context prepended. The definition 24 then can be copied to Workbench and formatted 23 for readability (select it, right click, Source 22 -> Format)

Clear[commonestContexts, contextFreeDefinition]

commonestContexts[sym_Symbol, n_: 1] := Quiet[
  Commonest[
   Cases[Level[DownValues[sym], {-1}, HoldComplete], 
    s_Symbol /; FreeQ[$ContextPath, Context[s]] :> Context[s]], n],
  Commonest::dstlms]

contextFreeDefinition::contexts = "Not showing the following contexts: `1`";

contextFreeDefinition[sym_Symbol, contexts_List] := 
 (If[contexts =!= {}, Message[contextFreeDefinition::contexts, contexts]];
  Internal`InheritedBlock[{sym}, ClearAttributes[sym, ReadProtected];
   Block[{$ContextPath = Join[$ContextPath, contexts]}, 
    Print@InputForm[FullDefinition[sym]]]])

contextFreeDefinition[sym_Symbol, context_String] := 
 contextFreeDefinition[sym, {context}]

contextFreeDefinition[sym_Symbol] := 
 contextFreeDefinition[sym, commonestContexts[sym]]

withRules[]

Caveat: This function does not localize 21 variables the same way With and Module do, which means 20 that nested localization constructs won't 19 work as expected. withRules[{a -> 1, b -> 2}, With[{a=3}, b_ :> b]] will replace a and b in the 18 nested With and Rule, while With doesn't do this.

This 17 is a variant of With that uses rules instead 16 of = and :=:

ClearAll[withRules]
SetAttributes[withRules, HoldAll]
withRules[rules_, expr_] :=
  Internal`InheritedBlock[
    {Rule, RuleDelayed},
    SetAttributes[{Rule, RuleDelayed}, HoldFirst];
    Unevaluated[expr] /. rules
  ]

I found this useful while cleaning 15 up code written during experimentation and 14 localizing variables. Occasionally I end 13 up with parameter lists in the form of {par1 -> 1.1, par2 -> 2.2}. With 12 withRules parameter values are easy to inject into 11 code previously written using global variables.

Usage 10 is just like With:

withRules[
  {a -> 1, b -> 2},
  a+b
]

Antialiasing 3D graphics

This is a very simple technique 9 to antialias 3D graphics even if your graphics 8 hardware doesn't support it natively.

antialias[g_, n_: 3] := 
  ImageResize[Rasterize[g, "Image", ImageResolution -> n 72], Scaled[1/n]]

Here's 7 an example:

Mathematica graphics Mathematica graphics

Note that a large value for 6 n or a large image size tends to expose graphics 5 driver bugs or introduce artefacts.


Notebook diff functionality

Notebook 4 diff functionality is available in the <<AuthorTools` package, and 3 (at least in version 8) in the undocumented 2 NotebookTools` context. This is a little GUI to diff 1 two notebooks that are currently open:

PaletteNotebook@DynamicModule[
  {nb1, nb2}, 
  Dynamic@Column[
    {PopupMenu[Dynamic[nb1], 
      Thread[Notebooks[] -> NotebookTools`NotebookName /@ Notebooks[]]], 
     PopupMenu[Dynamic[nb2], 
      Thread[Notebooks[] -> NotebookTools`NotebookName /@ Notebooks[]]], 
     Button["Show differences", 
      CreateDocument@NotebookTools`NotebookDiff[nb1, nb2]]}]
  ]

Mathematica graphics

Score: 9

Recursive pure functions (#0) seem to be one 32 of the darker corners of the language. Here 31 are a couple of non-trivial examples of 30 their use , where this is really useful 29 (not that they can not be done without it). The 28 following is a pretty concise and reasonably 27 fast function to find connected components 26 in a graph, given a list of edges specified 25 as pairs of vertices:

ClearAll[setNew, componentsBFLS];
setNew[x_, x_] := Null;
setNew[lhs_, rhs_]:=lhs:=Function[Null, (#1 := #0[##]); #2, HoldFirst][lhs, rhs];

componentsBFLS[lst_List] := Module[{f}, setNew @@@ Map[f, lst, {2}];
   GatherBy[Tally[Flatten@lst][[All, 1]], f]];

What happens here is 24 that we first map a dummy symbol on each 23 of the vertex numbers, and then set up a 22 way that, given a pair of vertices {f[5],f[10]}, say, then 21 f[5] would evaluate to f[10]. The recursive pure 20 function is used as a path compressor (to 19 set up memoization in such a way that instead 18 of long chains like f[1]=f[3],f[3]=f[4],f[4]=f[2], ..., memoized values get 17 corrected whenever a new "root" of the component 16 is discovered. This gives a significant 15 speed-up. Because we use assignment, we 14 need it to be HoldAll, which makes this 13 construct even more obscure and more attractive 12 ). This function is a result of on and off-line 11 Mathgroup discussion involving Fred Simons, Szabolcs 10 Horvat, DrMajorBob and yours truly. Example:

In[13]:= largeTest=RandomInteger[{1,80000},{40000,2}];

In[14]:= componentsBFLS[largeTest]//Short//Timing
Out[14]= {0.828,{{33686,62711,64315,11760,35384,45604,10212,52552,63986,  
     <<8>>,40962,7294,63002,38018,46533,26503,43515,73143,5932},<<10522>>}}

It 9 is certainly much slower than a built-in, but 8 for the size of code, quite fast still IMO.

Another 7 example: here is a recursive realization 6 of Select, based on linked lists and recursive 5 pure functions:

selLLNaive[x_List, test_] :=
  Flatten[If[TrueQ[test[#1]],
     {#1, If[#2 === {}, {}, #0 @@ #2]},
     If[#2 === {}, {}, #0 @@ #2]] & @@ Fold[{#2, #1} &, {}, Reverse[x]]];

For example,

In[5]:= Block[
         {$RecursionLimit= Infinity},
         selLLNaive[Range[3000],EvenQ]]//Short//Timing

Out[5]= {0.047,{2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,
 <<1470>>,2972,2974,2976,2978,2980,2982,2984,2986,2988,2990,
  2992,2994,2996,2998,3000}}

It is however 4 not properly tail recursive, and will blow 3 the stack (crash the kernel) for larger 2 lists. Here is the tail-recursive version:

selLLTailRec[x_List, test_] :=
Flatten[
 If[Last[#1] === {},
  If[TrueQ[test[First[#1]]],
   {#2, First[#1]}, #2],
  (* else *)
  #0[Last[#1],
   If[TrueQ[test[First[#1]]], {#2, First[#1]}, #2]
   ]] &[Fold[{#2, #1} &, {}, Reverse[x]], {}]];

For 1 example,

In[6]:= Block[{$IterationLimit= Infinity},
       selLLTailRec[Range[500000],EvenQ]]//Short//Timing
Out[6]= {2.39,{2,4,6,8,10,12,14,16,18,20,22,
       <<249978>>,499980,499982,499984,499986,499988,499990,499992,
        499994,499996,499998,500000}} 
Score: 8

This is recipe from Stan Wagon's book...use 7 it when built-in Plot behaves erratically 6 due to lack of precision

Options[PrecisePlot] = {PrecisionGoal -> 6};
PrecisePlot[f_, {x_, a_, b_}, opts___] := Module[{g, pg},
   pg = PrecisionGoal /. {opts} /. Options[PrecisePlot];
   SetAttributes[g, NumericFunction];
   g[z_?InexactNumberQ] := Evaluate[f /. x -> z];
   Plot[N[g[SetPrecision[y, \[Infinity]]], pg], {y, a, b},
    Evaluate[Sequence @@ FilterRules[{opts}, Options[Plot]]]]];

I often use the 5 following trick from Kristjan Kannike's 4 when I need "dictionary-like" behavior from 3 Mathematica's downvalues

index[downvalue_, 
   dict_] := (downvalue[[1]] /. HoldPattern[dict[x_]] -> x) // 
   ReleaseHold;
value[downvalue_] := downvalue[[-1]];
indices[dict_] := 
  Map[#[[1]] /. {HoldPattern[dict[x_]] -> x} &, DownValues[dict]] // 
   ReleaseHold;
values[dict_] := Map[#[[-1]] &, DownValues[dict]];
items[dict_] := Map[{index[#, dict], value[#]} &, DownValues[dict]];
indexQ[dict_, index_] := 
  If[MatchQ[dict[index], HoldPattern[dict[index]]], False, True];

(* Usage example: *)
(* Count number of times each subexpression occurs in an expression *)
expr = Cos[x + Cos[Cos[x] + Sin[x]]] + Cos[Cos[x] + Sin[x]]
Map[(counts[#] = If[indexQ[counts, #], counts[#] + 1, 1]; #) &, expr, Infinity];
items[counts]

When evaluation 2 results are confusing, sometimes it helps 1 to dump evaluation steps into a text file

SetAttributes[recordSteps, HoldAll];
recordSteps[expr_] :=
 Block[{$Output = List@OpenWrite["~/temp/msgStream.m"]}, 
  TracePrint[Unevaluated[expr], _?(FreeQ[#, Off] &), 
   TraceInternal -> True];
  Close /@ $Output;
  Thread[Union@
    Cases[ReadList["~/temp/msgStream.m", HoldComplete[Expression]], 
     symb_Symbol /; 
       AtomQ@Unevaluated@symb && 
        Context@Unevaluated@symb === "System`" :> 
      HoldComplete@symb, {0, Infinity}, Heads -> True], HoldComplete]
  ]

(* Usage example: *)
(* puts steps of evaluation of 1+2+Sin[5]) into ~/temp/msgStream.m *)
recordSteps[1+2+Sin[5]]
Score: 8

It is possible to run MathKernel in batch 30 mode by using undocumented command-line options -batchinput and -batchoutput:

math -batchinput -batchoutput < input.m > outputfile.txt

(where input.m is the 29 batch input file ending with the newline 28 character, outputfile.txt is the file to which the output 27 will be redirected).

In Mathematica v.>=6 the MathKernel 26 has undocumented command-line option:

-noicon

which 25 controls whether the MathKernel will have 24 visible icon on the Taskbar (at least under 23 Windows).

The FrontEnd (at least from v.5) has 22 undocumented command-line option

-b

which disables 21 the splash-screen and allows to run the 20 Mathematica FrontEnd much faster

and option

-directlaunch

which disables the mechanism which launches the most recent Mathematica version installed instead 19 of launching the version associated with 18 .nb files in the system registry.

Another 17 way to do this probably is:

Instead of launching 16 the Mathematica.exe binary in the installation 15 directory, launch the Mathematica.exe 14 binary in SystemFiles\FrontEnd\Binaries\Windows. The 13 former is a simple launcher program which 12 tries its hardest to redirect requests 11 for opening notebooks to running copies 10 of the user interface. The latter is 9 the user interface binary itself.

It is 8 handy to combine the last command line option 7 with setting global FrontEnd option VersionedPreferences->True which disables sharing of preferences between different Mathematica versions installed:

SetOptions[$FrontEnd, VersionedPreferences -> True]

(The 6 above should be evaluated in the most recent 5 Mathematica version installed.)

In Mathematica 8 this is controlled in the Preferences dialog, in the System pane, under the setting "Create and maintain version specific front end preferences".

It is possible 4 to get incomplete list of command-line options 3 of the FrontEnd by using undocumented key 2 -h (the code for Windows):

SetDirectory[$InstallationDirectory <> 
   "\\SystemFiles\\FrontEnd\\Binaries\\Windows\\"];
Import["!Mathematica -h", "Text"]

gives:

Usage:  Mathematica [options] [files]
Valid options:
    -h (--help):  prints help message
    -cleanStart (--cleanStart):  removes existing preferences upon startup
    -clean (--clean):  removes existing preferences upon startup
    -nogui (--nogui):  starts in a mode which is initially hidden
    -server (--server):  starts in a mode which disables user interaction
    -activate (--activate):  makes application frontmost upon startup
    -topDirectory (--topDirectory):  specifies the directory to search for resources and initialization files
    -preferencesDirectory (--preferencesDirectory):  specifies the directory to search for user AddOns and preference files
    -password (--password):  specifies the password contents
    -pwfile (--pwfile):  specifies the path for the password file
    -pwpath (--pwpath):  specifies the directory to search for the password file
    -b (--b):  launches without the splash screen
    -min (--min):  launches as minimized

Other options 1 include:

-directLaunch:  force this FE to start
-32:  force the 32-bit FE to start
-matchingkernel:  sets the frontend to use the kernel of matching bitness
-Embedding:  specifies that this instance is being used to host content out of process

Are there other potentially useful command-line options of the MathKernel and the FrontEnd? Please share if you know.

Related question.

Score: 7

My favorite hacks are small code-generating 25 macros that allow you to replace a bunch 24 of standard boilerplate commands with one 23 short one. Alternatively, you can create 22 commands for opening/creating notebooks.

Here 21 is what I've been using for a while in my 20 day-to-day Mathematica workflow. I found 19 myself performing the following a lot:

  1. Make a notebook have a private context, load package(s) I need, make it autosave.
  2. After working with this notebook for a while, I'd want to do some throw away scratch computations in a separate notebook, with its own private context, while having access to definitions I've been using in the "main" notebook. Because I set up the private context, this requires to manually adjust $ContextPath

Doing 18 all this by hand over and over is a pain, so 17 let's automate! First, some utility code:

(* Credit goes to Sasha for SelfDestruct[] *)
SetAttributes[SelfDestruct, HoldAllComplete];
SelfDestruct[e_] := (If[$FrontEnd =!= $Failed,
   SelectionMove[EvaluationNotebook[], All, EvaluationCell]; 
   NotebookDelete[]]; e)

writeAndEval[nb_,boxExpr_]:=(
    NotebookWrite[nb,  CellGroupData[{Cell[BoxData[boxExpr],"Input"]}]];
    SelectionMove[nb, Previous, Cell]; 
    SelectionMove[nb, Next, Cell];
    SelectionEvaluate[nb];
)

ExposeContexts::badargs = 
  "Exposed contexts should be given as a list of strings.";
ExposeContexts[list___] := 
 Module[{ctList}, ctList = Flatten@List@list; 
  If[! MemberQ[ctList, Except[_String]],AppendTo[$ContextPath, #] & /@ ctList, 
   Message[ExposeContexts::badargs]];
  $ContextPath = DeleteDuplicates[$ContextPath];
  $ContextPath]

    Autosave[x:(True|False)] := SetOptions[EvaluationNotebook[],NotebookAutoSave->x];

Now, let's 16 create a macro that's going to put the following 15 cells in the notebook:

SetOptions[EvaluationNotebook[], CellContext -> Notebook]
Needs["LVAutils`"]
Autosave[True]

And here's the macro:

MyPrivatize[exposedCtxts : ({__String} | Null) : Null]:=
  SelfDestruct@Module[{contBox,lvaBox,expCtxtBox,assembledStatements,strList},
    contBox = MakeBoxes[SetOptions[EvaluationNotebook[], CellContext -> Notebook]];
    lvaBox = MakeBoxes[Needs["LVAutils`"]];

    assembledStatements = {lvaBox,MakeBoxes[Autosave[True]],"(*********)"};
    assembledStatements = Riffle[assembledStatements,"\[IndentingNewLine]"]//RowBox;
    writeAndEval[InputNotebook[],contBox];
    writeAndEval[InputNotebook[],assembledStatements];
    If[exposedCtxts =!= Null,
       strList = Riffle[("\"" <> # <> "\"") & /@ exposedCtxts, ","];
       expCtxtBox = RowBox[{"ExposeContexts", "[", RowBox[{"{", RowBox[strList], "}"}], "]"}];
       writeAndEval[InputNotebook[],expCtxtBox];
      ]
 ]

Now 14 when I type in MyPrivatize[] is creates the private context 13 and loads my standard package. Now let's 12 create a command that will open a new scratch 11 notebook with its own private context (so 10 that you can hack there with wild abandon 9 without the risk of screwing up the definitions), but 8 has access to your current contexts.

SpawnScratch[] := SelfDestruct@Module[{nb,boxExpr,strList},
    strList = Riffle[("\"" <> # <> "\"") & /@ $ContextPath, ","];
    boxExpr = RowBox[{"MyPrivatize", "[",
        RowBox[{"{", RowBox[strList], "}"}], "]"}];
    nb = CreateDocument[];
    writeAndEval[nb,boxExpr];
]

The 7 cool thing about this is that due to SelfDestruct, when 6 the command runs it leaves no trace in the 5 current notebook -- which is good, because 4 otherwise it would just create clutter.

For 3 extra style points, you can create keyword 2 triggers for these macros using InputAutoReplacements, but I'll 1 leave this as an exercise for the reader.

Score: 7

PutAppend with PageWidth -> Infinity

In Mathematica using of the PutAppend command is the most straightforward 20 way to maintain a running log file with 19 results of intermediate computations. But 18 it uses by default PageWith->78 setting when exporting 17 expressions to a file and so there is no 16 guarantee that every intermediate output 15 will take only one line in the log.

PutAppend does 14 not have any options itself but tracing 13 its evaluations reveals that it is based 12 on the OpenAppend function which has the PageWith option and 11 allows changing its default value by the 10 SetOptions command:

In[2]:= Trace[x>>>"log.txt",TraceInternal->True]
Out[2]= {x>>>log.txt,{OpenAppend[log.txt,CharacterEncoding->PrintableASCII],OutputStream[log.txt,15]},Null}

So we can get PutAppend to append only one 9 line at a time by setting:

SetOptions[OpenAppend, PageWidth -> Infinity]

UPDATE

There is a bug introduced 8 in version 10 (fixed in version 11.3): SetOptions no 7 longer affects the behavior of OpenWrite and OpenAppend.

A 6 workaround is to implement your own version 5 of PutAppend with explicit PageWidth -> Infinity option:

Clear[myPutAppend]
myPutAppend[expr_, pathtofile_String] :=
 (Write[#, expr]; Close[#];) &[OpenAppend[pathtofile, PageWidth -> Infinity]]

Note that we also 4 may implement it via WriteString as shown in this answer, but 3 in this case it will be necessary to preliminarily 2 convert the expression into the corresponding 1 InputForm via ToString[expr, InputForm].

Score: 6

I was just looking through one of my packages 7 for inclusion in this, and found some messages 6 that I defined that work wonders: Debug::<some name>. By 5 default, they are turned off, so don't produce 4 much overhead. But, I can litter my code 3 with them, and turn them on if I need to 2 figure out exactly how a bit of code is 1 behaving.

Score: 6

One of the things that bothers me about 16 the built-in scoping constructs is that 15 they evaluate all of the local variable 14 definitions at once, so you can't write 13 for example

With[{a = 5, b = 2 * a},
    ...
]

So a while ago I came up with 12 a macro called WithNest that allows you 11 to do this. I find it handy, since it lets 10 you keep variable bindings local without 9 having to do something like

Module[{a = 5,b},
    b = 2 * a;
    ...
]

In the end, the 8 best way I could find to do this was by 7 using a special symbol to make it easier 6 to recurse over the list of bindings, and 5 I put the definition into its own package 4 to keep this symbol hidden. Maybe someone 3 has a simpler solution to this problem?

If 2 you want to try it out, put the following 1 into a file called Scoping.m:

BeginPackage["Scoping`"];

WithNest::usage=
"WithNest[{var1=val1,var2=val2,...},body] works just like With, except that
values are evaluated in order and later values have access to earlier ones.
For example, val2 can use var1 in its definition.";

Begin["`Private`"];

(* Set up a custom symbol that works just like Hold. *)
SetAttributes[WithNestHold,HoldAll];

(* The user-facing call.  Give a list of bindings and a body that's not
our custom symbol, and we start a recursive call by using the custom
symbol. *)
WithNest[bindings_List,body:Except[_WithNestHold]]:=
WithNest[bindings,WithNestHold[body]];

(* Base case of recursive definition *)
WithNest[{},WithNestHold[body_]]:=body;

WithNest[{bindings___,a_},WithNestHold[body_]]:=
WithNest[
{bindings},
WithNestHold[With[List@a,body]]];

SyntaxInformation[WithNest]={"ArgumentsPattern"->{{__},_}};
SetAttributes[WithNest,{HoldAll,Protected}];

End[];

EndPackage[];
Score: 5

This one was written by Alberto Di Lullo, (who 6 doesn't appear to be on Stack Overflow).

CopyToClipboard, for 5 Mathematica 7 (in Mathematica 8 it's built 4 in)

CopyToClipboard[expr_] := 
  Module[{nb}, 
   nb = CreateDocument[Null, Visible -> False, WindowSelected -> True];
   NotebookWrite[nb, Cell[OutputFormData@expr], All];
   FrontEndExecute[FrontEndToken[nb, "Copy"]];
   NotebookClose@nb];

Original post: http://forums.wolfram.com/mathgroup/archive/2010/Jun/msg00148.html

I have found this routine 3 useful for copying large real numbers to 2 the clipboard in ordinary decimal form. E.g. CopyToClipboard["123456789.12345"]

Cell[OutputFormData@expr] neatly 1 removes the quotes.

Score: 5

This code makes a palette that uploads the 8 selection to Stack Exchange as an image. On 7 Windows, an extra button is provided that 6 gives a more faithful rendering of the selection.

Copy 5 the code into a notebook cell and evaluate. Then 4 pop out the palette from the output, and 3 install it using Palettes -> Install Palette...

If you have any trouble 2 with it, post a comment here. Download the 1 notebook version here.


Begin["SOUploader`"];

Global`palette = PaletteNotebook@DynamicModule[{},

   Column[{
     Button["Upload to SE",
      With[{img = rasterizeSelection1[]},
       If[img === $Failed, Beep[], uploadWithPreview[img]]],
      Appearance -> "Palette"],

     If[$OperatingSystem === "Windows",

      Button["Upload to SE (pp)",
       With[{img = rasterizeSelection2[]},
        If[img === $Failed, Beep[], uploadWithPreview[img]]],
       Appearance -> "Palette"],

      Unevaluated@Sequence[]
      ]
     }],

   (* Init start *)
   Initialization :>
    (

     stackImage::httperr = "Server returned respose code: `1`";
     stackImage::err = "Server returner error: `1`";

     stackImage[g_] :=
      Module[
       {getVal, url, client, method, data, partSource, part, entity,
        code, response, error, result},

       getVal[res_, key_String] :=
        With[{k = "var " <> key <> " = "},
         StringTrim[

          First@StringCases[
            First@Select[res, StringMatchQ[#, k ~~ ___] &],
            k ~~ v___ ~~ ";" :> v],
          "'"]
         ];

       data = ExportString[g, "PNG"];

       JLink`JavaBlock[
        url = "http://stackoverflow.com/upload/image";
        client =
         JLink`JavaNew["org.apache.commons.httpclient.HttpClient"];
        method =
         JLink`JavaNew[
          "org.apache.commons.httpclient.methods.PostMethod", url];
        partSource =
         JLink`JavaNew[
          "org.apache.commons.httpclient.methods.multipart.\
ByteArrayPartSource", "mmagraphics.png",
          JLink`MakeJavaObject[data]@toCharArray[]];
        part =
         JLink`JavaNew[
          "org.apache.commons.httpclient.methods.multipart.FilePart",
          "name", partSource];
        part@setContentType["image/png"];
        entity =
         JLink`JavaNew[
          "org.apache.commons.httpclient.methods.multipart.\
MultipartRequestEntity", {part}, method@getParams[]];
        method@setRequestEntity[entity];
        code = client@executeMethod[method];
        response = method@getResponseBodyAsString[];
        ];

       If[code =!= 200, Message[stackImage::httperr, code];
        Return[$Failed]];
       response = StringTrim /@ StringSplit[response, "\n"];

       error = getVal[response, "error"];
       result = getVal[response, "result"];
       If[StringMatchQ[result, "http*"],
        result,
        Message[stackImage::err, error]; $Failed]
       ];

     stackMarkdown[g_] :=
      "![Mathematica graphics](" <> stackImage[g] <> ")";

     stackCopyMarkdown[g_] := Module[{nb, markdown},
       markdown = Check[stackMarkdown[g], $Failed];
       If[markdown =!= $Failed,
        nb = NotebookCreate[Visible -> False];
        NotebookWrite[nb, Cell[markdown, "Text"]];
        SelectionMove[nb, All, Notebook];
        FrontEndTokenExecute[nb, "Copy"];
        NotebookClose[nb];
        ]
       ];

     (* Returns available vertical screen space,
     taking into account screen elements like the taskbar and menu *)


     screenHeight[] := -Subtract @@
        Part[ScreenRectangle /. Options[$FrontEnd, ScreenRectangle],
         2];

     uploadWithPreview[img_Image] :=
      CreateDialog[
       Column[{
         Style["Upload image to the Stack Exchange network?", Bold],
         Pane[

          Image[img, Magnification -> 1], {Automatic,
           Min[screenHeight[] - 140, 1 + ImageDimensions[img][[2]]]},
          Scrollbars -> Automatic, AppearanceElements -> {},
          ImageMargins -> 0
          ],
         Item[
          ChoiceButtons[{"Upload and copy MarkDown"}, \
{stackCopyMarkdown[img]; DialogReturn[]}], Alignment -> Right]
         }],
       WindowTitle -> "Upload image to Stack Exchange?"
       ];

     (* Multiplatform, fixed-width version.
        The default max width is 650 to fit Stack Exchange *)
     rasterizeSelection1[maxWidth_: 650] :=
      Module[{target, selection, image},
       selection = NotebookRead[SelectedNotebook[]];
       If[MemberQ[Hold[{}, $Failed, NotebookRead[$Failed]], selection],

        $Failed, (* There was nothing selected *)

        target =
         CreateDocument[{}, WindowSelected -> False, Visible -> False,
           WindowSize -> maxWidth];
        NotebookWrite[target, selection];
        image = Rasterize[target, "Image"];
        NotebookClose[target];
        image
        ]
       ];

     (* Windows-only pixel perfect version *)
     rasterizeSelection2[] :=
      If[
       MemberQ[Hold[{}, $Failed, NotebookRead[$Failed]],
        NotebookRead[SelectedNotebook[]]],

       $Failed, (* There was nothing selected *)

       Module[{tag},
        FrontEndExecute[
         FrontEndToken[FrontEnd`SelectedNotebook[], "CopySpecial",
          "MGF"]];
        Catch[
         NotebookGet@ClipboardNotebook[] /.
          r_RasterBox :>
           Block[{},
            Throw[Image[First[r], "Byte", ColorSpace -> "RGB"], tag] /;
              True];
         $Failed,
         tag
         ]
        ]
       ];
     )
   (* Init end *)
   ]

End[];
Score: 4

I'm sure a lot of people have encountered 33 the situation where they run some stuff, realizing 32 it not only stuck the program, but they 31 also haven't saved for the last 10 minutes!

EDIT

After suffering from this for some 30 time, I one day found out that one can create auto-save from within 29 the Mathematica code. I think that using such auto-save 28 have helped me a lot in the past, and I 27 always felt that the possibility itself was something that not 26 a lot of people are aware that they can 25 do.

The original code I used is at the bottom. Thanks 24 to the comments I've found out that it is 23 problematic, and that it is much better 22 to do it in an alternative way, using ScheduledTask (which 21 will work only in Mathematica 8).

Code for this can 20 be found in this answer from Sjoerd C. de Vries (Since I'm not sure if 19 it's OK to copy it to here, I'm leaving 18 it as a link only.)


The solution below is 17 using Dynamic. It will save the notebook every 16 60 seconds, but apparently only if its cell is visible. I'm leaving 15 it here only for completion reasons. (and 14 for users of Mathematica 6 and 7)

/EDIT

To solve it I use 13 this code in the beginning of a notebook:

Dynamic[Refresh[NotebookSave[]; DateString[], UpdateInterval -> 60]]

This 12 will save your work every 60 seconds.
I 11 prefer it to NotebookAutoSave[] because it saves before the 10 input is processed, and because some files 9 are more text than input.

I originally found 8 it here: http://en.wikipedia.org/wiki/Talk:Mathematica#Criticisms

Note that once running this line, saving 7 will happen even if you close and re-open 6 your file (as long as dynamic updating is 5 enabled).

Also, since there is no undo in 4 Mathematica, be careful not to delete all your content, since 3 saving will make it irreversible (as a precaution 2 move, I remove this code from every finished 1 notebook)

Score: 3

Remember that The Mathematica Book is also 2 available online at http://reference.wolfram.com/legacy/v5_2/ - though it's superseded 1 by the current documentation at http://reference.wolfram.com

Score: 3

I find it really useful when developing 15 packages to add this keyboard shortcut to 14 my SystemFiles/FrontEnd/TextResources/Windows/KeyEventTranslations.tr file.

(* Evaluate Initialization Cells: Real useful for reloading library changes. *)

Item[KeyEvent["i", Modifiers -> {Control, Command}],
    FrontEndExecute[
        FrontEndToken[
            SelectedNotebook[],
            "EvaluateInitialization"]]],

Next for every Packagename.m I make a PackagenameTest.nb notebook 13 for testing and the first 2 cells of the 12 test notebook are set as initialization 11 cells. In the first cell I put

Needs["PackageManipulations`"]

to load the 10 very useful PackageManipulations library which was written by 9 Leonid. The second cell contains

PackageRemove["Packagename`Private`"]
PackageRemove["Packagename`"]
PackageReload["Packagename`"]

which all 8 do the actual package reloading. Note the 7 first two lines are there only to Remove all symbols 6 as I like to keep the contexts as clean 5 as possible.

Then the workflow for writing 4 and testing a package becomes something 3 like this.

  1. Save changes to Packagename.m.
  2. Go to PackagenameTest.nb and do CTRL + ALT + i.

This causes the initialization 2 cells to reload the package, which makes 1 testing real simple.

Score: 1

Following function format[expr_] can be used to indent/format 2 unformatted mathematica expressions that spans over 1 a page

indent[str_String, ob_String, cb_String, delim_String] := 
  Module[{ind, indent, f, tab}, ind = 0; tab = "    ";
   indent[i_, tab_, nl_] := nl <> Nest[tab <> ToString[#] &, "", i];
   f[c_] := (indent[ind, "", " "] <> c <> indent[++ind, tab, "\n"]) /;StringMatchQ[ob, ___ ~~ c ~~ ___];
   f[c_] := (indent[--ind, "", " "] <> c <> indent[ind, tab, "\n"]) /;StringMatchQ[cb, ___ ~~ c ~~ ___];
   f[c_] := (c <> indent[ind, tab, "\n"]) /;StringMatchQ[delim, ___ ~~ c ~~ ___];
   f[c_] := c;
   f /@ Characters@str // StringJoin];
format[expr_] := indent[expr // InputForm // ToString, "[({", "])}", ";"];

(*    
format[Hold@Module[{ind, indent, f, tab}, ind = 0; tab = "    ";
 indent[i_, tab_, nl_] := nl <> Nest[tab <> ToString[#] &, "", i];
 f[c_] := (indent[ind, "", " "] <> c <> indent[++ind, tab, "\n"]) /;StringMatchQ[ob, ___ ~~ c ~~ ___];
 f[c_] := (indent[--ind, "", " "] <> c <> indent[ind, tab, "\n"]) /;StringMatchQ[cb, ___ ~~ c ~~ ___];
 f[c_] := (c <> indent[ind, tab, "\n"]) /;StringMatchQ[delim, ___ ~~ c ~~ ___];
 f[c_] := c;
 f /@ Characters@str // StringJoin]]
*)

ref: https://codegolf.stackexchange.com/questions/3088/indent-a-string-using-given-parentheses

More Related questions