Professional Automated Trading

An insider's view of how to develop and operate an automated proprietary trading network

Reflecting author Eugene Durenard's extensive experience in this field, Professional Automated Trading offers valuable insights you won't find anywhere else. It reveals how a series of concepts and techniques coming from current research in artificial life and modern control theory can be applied to the design of effective trading systems that outperform the majority of published trading systems. It also skillfully provides you with essential information on the practical coding and implementation of a scalable systematic trading architecture.

Based on years of practical experience in building successful research and infrastructure processes for purpose of trading at several frequencies, this book is designed to be a comprehensive guide for understanding the theory of design and the practice of implementation of an automated systematic trading process at an institutional scale.
* Discusses several classical strategies and covers the design of efficient simulation engines for back and forward testing
* Provides insights on effectively implementing a series of distributed processes that should form the core of a robust and fault-tolerant automated systematic trading architecture
* Addresses trade execution optimization by studying market-pressure models and minimization of costs via applications of execution algorithms
* Introduces a series of novel concepts from artificial life and modern control theory that enhance robustness of the systematic decision making--focusing on various aspects of adaptation and dynamic optimal model choice

Engaging and informative, Proprietary Automated Trading covers the most important aspects of this endeavor and will put you in a better position to excel at it.

Preface xv
Chapter 1 Introduction to Systematic Trading 1
1. 1 Definition of Systematic Trading 2
1. 2 Philosophy of Trading 3
1. 2. 1 Lessons from the Market 3
1. 2. 2 Mechanism vs. Organism 5
1. 2. 3 The Edge of Complexity 5
1. 2. 4 Is Systematic Trading Reductionistic? 6
1. 2. 5 Reaction vs. Proaction 6
1. 2. 6 Arbitrage? 7
1. 2. 7 Two Viable Paths 7
1. 3 The Business of Trading 7
1. 3. 1 Profitability and Track Record 8
1. 3. 2 The Product and Its Design 10
1. 3. 3 The Trading Factory 12
1. 3. 4 Marketing and Distribution 15
1. 3. 5 Capital, Costs, and Critical Mass 16
1. 4 Psychology and Emotions 19
1. 4. 1 Ups and Downs 19
1. 4. 2 Peer Pressure and the Blame Game 20
1. 4. 3 Trust: Continuity of Quality 20
1. 4. 4 Learning from Each Other 21
1. 5 From Candlesticks in Kyoto to FPGAs in Chicago 22
Part One Strategy Design and Testing
Chapter 2 A New Socioeconomic Paradigm 33
2. 1 Financial Theory vs. Market Reality 33
2. 1. 1 Adaptive Reactions vs. Rigid Anticipations 33
2. 1. 2 Accumulation vs. Divestment Games 37
2. 1. 3 Phase Transitions under Leverage 38
2. 1. 4 Derivatives: New Risks Do Not Project onto Old Hedges 40
2. 1. 5 Socio-Political Dynamics and Feedbacks 41
2. 2 The Market Is a Complex Adaptive System 42
2. 2. 1 Emergence 43
2. 2. 2 Intelligence Is Not Always Necessary 44
2. 2. 3 The Need to Adapt 45
2. 3 Origins of Robotics and Artificial Life 45
Chapter 3 Analogies between Systematic Trading and Robotics 49
3. 1 Models and Robots 49
3. 2 The Trading Robot 50
3. 3 Finite-State-Machine Representation of the Control System 52
Chapter 4 Implementation of Strategies as Distributed Agents 57
4. 1 Trading Agent 57
4. 2 Events 60
4. 3 Consuming Events 60
4. 4 Updating Agents 61
4. 5 Defining FSM Agents 63
4. 6 Implementing a Strategy 66
Chapter 5 Inter-Agent Communications 73
5. 1 Handling Communication Events 73
5. 2 Emitting Messages and Running Simulations 75
5. 3 Implementation Example 76
Chapter 6 Data Representation Techniques 83
6. 1 Data Relevance and Filtering of Information 83
6. 2 Price and Order Book Updates 84
6. 2. 1 Elementary Price Events 85
6. 2. 2 Order Book Data 85
6. 2. 3 Tick Data: The Finest Grain 88
6. 3 Sampling: Clock Time vs. Event Time 89
6. 4 Compression 90
6. 4. 1 Slicing Time into Bars and Candles 90
6. 4. 2 Slicing Price into Boxes 96
6. 4. 3 Market Distributions 97
6. 5 Representation 97
6. 5. 1 Charts and Technical Analysis 99
6. 5. 2 Translating Patterns into Symbols 101
6. 5. 3 Translating News into Numbers 102
6. 5. 4 Psychology of Data and Alerts 104
Chapter 7 Basic Trading Strategies 105
7. 1 Trend-Following 105
7. 1. 1 Channel Breakout 106
7. 1. 2 Moving Averages 106
7. 1. 3 Swing Breakout 112
7. 2 Acceleration 114
7. 2. 1 Trend Asymmetry 115
7. 2. 2 The Shadow Index 116
7. 2. 3 Trading Acceleration 117
7. 3 Mean-Reversion 118
7. 3. 1 Swing Reversal 118
7. 3. 2 Range Projection 120
7. 4 Intraday Patterns 122
7. 4. 1 Openings 122
7. 4. 2 Seasonality of Volatility 122
7. 5 News-Driven Strategies 124
7. 5. 1 Expectations vs. Reality 124
7. 5. 2 Ontology-Driven Strategies 125
Chapter 8 Architecture for Market-Making 127
8. 1 Traditional Market-Making: The Specialists 127
8. 2 Conditional Market-Making: Open Outcry 128
8. 3 Electronic Market-Making 129
8. 4 Mixed Market-Making Model 131
8. 5 An Architecture for a Market-Making Desk 134
Chapter 9 Combining Strategies into Portfolios 139
9. 1 Aggregate Agents 139
9. 2 Optimal Portfolios 141
9. 3 Risk-Management of a Portfolio of Models 142
Chapter 10 Simulating Agent-Based Strategies 145
10. 1 The Simulation Problem 146
10. 2 Modeling the Order Management System 147
10. 2. 1 Orders and Algorithms 148
10. 2. 2 Simulating Slippage 149
10. 2. 3 Simulating Order Placement 151
10. 2. 4 Simulating Order Execution 153
10. 2. 5 A Model for the OMS 155
10. 2. 6 Operating the OMS 156
10. 3 Running Simulations 158
10. 3. 1 Setting Up a Back Test 158
10. 3. 2 Setting Up a Forward Test 160
10. 4 Analysis of Results 162
10. 4. 1 Continuous Statistics 163
10. 4. 2 Per-Trade Statistics 164
10. 4. 3 Parameter Search and Optimization 165
10. 5 Degrees of Over-Fitting 167
Part Two Evolving Strategies
Chapter 11 Strategies for Adaptation 173
11. 1 Avenues for Adaptations 173
11. 2 The Cybernetics of Trading 175
Chapter 12 Feedback and Control 179
12. 1 Looking at Markets through Models 179
12. 1. 1 Internal World 179
12. 1. 2 Strategies as Generalized Filters 180
12. 1. 3 Implicit Market Regimes 181
12. 1. 4 Persistence of Regimes 183
12. 2 Fitness Feedback Control 184
12. 2. 1 Measures of Fitness 186
12. 3 Robustness of Strategies 192
12. 4 Efficiency of Control 193
12. 4. 1 Triggering Control 193
12. 4. 2 Measuring Efficiency of Control 194
12. 4. 3 Test Results 196
12. 4. 4 Optimizing Control Parameters 197
Chapter 13 Simple Swarm Systems 199
13. 1 Switching Strategies 199
13. 1. 1 Switching between Regimes 200
13. 1. 2 Switching within the Same Regime 200
13. 1. 3 Mechanics of Switching and Transaction Costs 205
13. 2 Strategy Neighborhoods 206
13. 3 Choice of a Simple Individual from a Population 208
13. 4 Additive Swarm System 210
13. 4. 1 Example of an Additive Swarm 211
13. 5 Maximizing Swarm System 214
13. 5. 1 Example of a Maximizing Swarm 215
13. 6 Global Performance Feedback Control 216
Chapter 14 Implementing Swarm Systems 219
14. 1 Setting Up the Swarm Strategy Set 220
14. 2 Running the Swarm 220
Chapter 15 Swarm Systems with Learning 223
15. 1 Reinforcement Learning 224
15. 2 Swarm Efficiency 224
15. 3 Behavior Exploitation by the Swarm 225
15. 4 Exploring New Behaviors 227
15. 5 Lamark among the Machines 227
Part Three Optimizing Execution
Chapter 16 Analysis of Trading Costs 231
16. 1 No Free Lunch 231
16. 2 Slippage 232
16. 3 Intraday Seasonality of Liquidity 233
16. 4 Models of Market Impact 234
16. 4. 1 Reaction to Aggression 235
16. 4. 2 Limits to Openness 235
Chapter 17 Estimating Algorithmic Execution Tools 237
17. 1 Basic Algorithmic Execution Tools 237
17. 2 Estimation of Algorithmic Execution Methodologies 240
17. 2. 1 A Simulation Engine for Algos 240
17. 2. 2 Using Execution Algo Results in Model Estimation 241
17. 2. 3 Joint Testing of Models and Algos 242
Part Four Practical Implementation
Chapter 18 Overview of a Scalable Architecture 247
18. 1 ECNs and Translation 247
18. 2 Aggregation and Disaggregation 249
18. 3 Order Management 250
18. 4 Controls 250
18. 5 Decisions 251
18. 6 Middle and Back Office 251
18. 7 Recovery 252
Chapter 19 Principal Design Patterns 253
19. 1 Language-Agnostic Domain Model 253
19. 2 Solving Tasks in Adapted Languages 254
19. 3 Communicating between Components 257
19. 3. 1 Messaging Bus 258
19. 3. 2 Remote Procedure Calls 259
19. 4 Distributed Computing and Modularity 260
19. 5 Parallel Processing 262
19. 6 Garbage Collection and Memory Control 263
Chapter 20 Data Persistence 265
20. 1 Business-Critical Data 265
20. 2 Object Persistence and Cached Memory 267
20. 3 Databases and Their Usage 269
Chapter 21 Fault Tolerance and Recovery Mechanisms 273
21. 1 Situations of Stress 273
21. 1. 1 Communication Breakdown 273
21. 1. 2 External Systems Breakdown 274
21. 1. 3 Trades Busted at the ECN Level 275
21. 1. 4 Give-Up Errors Causing Credit Line Problems 276
21. 1. 5 Internal Systems Breakdown 277
21. 1. 6 Planned Maintenance and Upgrades 277
21. 2 A Jam of Logs Is Better Than a Logjam of Errors 277
21. 3 Virtual Machine and Network Monitoring 278
Chapter 22 Computational Efficiency 281
22. 1 CPU Spikes 281
22. 2 Recursive Computation of Model Signals and Performance 282
22. 3 Numeric Efficiency 285
Chapter 23 Connectivity to Electronic Commerce Networks 291
23. 1 Adaptors 291
23. 2 The Translation Layer 292
23. 2. 1 Orders: FIX 292
23. 2. 2 Specific ECNs 293
23. 2. 3 Price Sources: FAST 293
23. 3 Dealing with Latency 294
23. 3. 1 External Constraints and Co-Location 294
23. 3. 2 Avoid Being Short the Latency Option 295
23. 3. 3 Synchronization under Constraints 296
23. 3. 4 Improving Internal Latency 297
Chapter 24 The Aggregation and Disaggregation Layer 299
24. 1 Quotes Filtering and Book Aggregation 300
24. 1. 1 Filtering Quotes 300
24. 1. 2 Synthetic Order Book 301
24. 2 Orders Aggregation and Fills Disaggregation 301
24. 2. 1 Aggregating Positions and Orders 301
24. 2. 2 Fills Disaggregation 303
24. 2. 3 Book Transfers and Middle Office 303
Chapter 25 The OMS Layer 305
25. 1 Order Management as a Recursive Controller 305
25. 1. 1 Management of Positions 307
25. 1. 2 Management of Resting Orders 307
25. 1. 3 Algorithmic Orders 308
25. 2 Control under Stress 309
25. 3 Designing a Flexible OMS 310
Chapter 26 The Human Control Layer 311
26. 1 Dashboard and Smart Scheduler 311
26. 1. 1 Parameter Control 311
26. 1. 2 Scheduled Flattening of Exposure 312
26. 2 Manual Orders Aggregator 313
26. 2. 1 Representing a Trader by an Agent 313
26. 2. 2 Writing a Trading Screen 314
26. 2. 3 Monitoring Aggregated Streams 314
26. 3 Position and P & L Monitor 314
26. 3. 1 Real-Time Exposure Monitor 315
26. 3. 2 Displaying Equity Curves 315
26. 3. 3 Online Trade Statistics and Fitnesses 315
26. 3. 4 Trades Visualization Module 317
Chapter 27 The Risk Management Layer 319
27. 1 Risky Business 319
27. 2 Automated Risk Management 320
27. 3 Manual Risk Control and the Panic Button 320
Chapter 28 The Core Engine Layer 323
28. 1 Architecture 323
28. 2 Simulation and Recovery 325
Chapter 29 Some Practical Implementation Aspects 327
29. 1 Architecture for Build and Patch Releases 327
29. 1. 1 Testing of Code before a Release 327
29. 1. 2 Versioning of Code and Builds 328
29. 1. 3 Persistence of State during Version Releases 328
29. 2 Hardware Considerations 329
29. 2. 1 Bottleneck Analysis 329
29. 2. 2 The Edge of Technology 330
Appendix Auxiliary LISP Functions 333
Bibliography 341
Index 351