The last time Hackerfall tried to access this page, it returned a not found error. A cached version of the page is below, or clickhereto continue anyway

Ex Base

ExBase

is a Wiki-friendly re-wording for "XBase", which is a general term for languages and tools derived from the Ashton Tate dBASE product, popular in the 1980s and early 1990s. Clones include

FoxPro

, Quicksilver, dbXL, and a compiled dialect called Clipper (which spawned

XbasePlusPlus

,

FlagShip

, and Harbour). XBase originated as a table-oriented non-SQL language and tool vaguely influenced by relational theory and (mainly) cursor-oriented table navigation techniques from the 1960s. It was one of the first products that easily allowed small businesses access to database concepts such as ad-hoc queries, indexing, and data-driven

CrudScreen

RAD when microcomputers were just coming of age. Jim "Button" Knopf's "PC-File" exhibited similar capabilities, albeit with less programmability.

ExBase

is kind of the Neil Diamond of programming tools: one tends to either have fond memories of it or nightmares about it, with little in between.

It allegedly has its roots in a legacy database product called RETRIEVE. The Pasadena Jet Propulsion Laboratory, famous for planetary probes, didn't want to pay licensing fees for RETRIEVE and so set out to create something similar. In a round-about way, it eventually wound up a commercial product for CPM microcomputers, and grew quite popular in the DOS world. Much of the history of the product was discovered during court testimony related to lawsuits by Ashton Tate to prevent clones of the product/tool. It turned out that Ashton Tate's view of history was not entirely accurate, and the history of

ExBase

was deeper than their original story. The judge eventually tossed the case, but the damage caused by the risk of legal claims on the language hurt the

ExBase

industry.

Its demise in popularity has been attributed to one or more of the following:

Other products such as Visual Basic, Microsoft Access, Paradox, and

PowerBuilder

eventually replaced its niche. Some feel the XBase vendors should have expanded on its table-oriented nature and used

ControlTable

s for event handling and GUI properties rather than try to copy OOP techniques popular in other GUI tools. An opportunity to go down a unique path was allegedly lost, and instead they tried to be a me-too product in an arena that was too different from the root table-oriented philosophy of the tool. You can turn a leopard into a tiger, but not into a gazelle. (

FoxPro

and

VisualFoxPro

actually use

ControlTable

s for many GUI aspects, but it is generally an

UndocumentedFeature

.)

The language gives one the feel of being "inside" database tables. It is heavily context-based because of its interactive origins. The plus side of being context-based is that less code is usually needed to indicate what is being operated on, and it made it easy to learn and debug using the interactive mode. The downside is that if not used carefully in longer modules, it can cause great confusion and scoping surprises.

An interesting feature is that iteration over a table was built into the language. A general form of the syntax is:

  do_x             // operate on current record
  do_x all         // operate on all records
  do_x for <expression>  // operate on records matching a "filter" expression

(Actually "&&" is used for comments instead of slashes, but that might confuse the reader here. Also, not all dialects allowed such on user-defined routines, but only built-in ones.)

It's essentially a

MapFunction

with predicate filtering (similar to an SQL "WHERE" clause).

So ExBase supports a form of HigherOrderFunction. Top, I'm surprised you're so opposed to them. Or are you opposed to this facility for the same reasons?

  • I'm not inherently opposed to them, I'm opposed to the way you propose using them. It's been pointed out many times that databases have FP-like features, and a lot of what would be loop-centric processing is handed over to the database engine (such as an RDBMS) instead of done in the app code.

  • No one has proposed replacing DBMS facilities with -- or avoiding them in favour of -- "loop-centric processing", have they? If that's your interpretation, I think you may have misunderstood something.

  • I haven't yet figured out the other side's stance. I'm still probing that one. It's not easy without sufficient realistic scenarios.

If you needed the same filter on multiple operations, then

FoxPro

and some other dialects introduced the "scan" construct:

  scan for <expression>
    do_something
    do_something_2
  endscan

One could also apply a filter for multiple statements:

  use <table>    // open a table
  set filter to <expression>
  do_something
  do_something_2
  scan
    do_something_3
  endscan
  etc...
  set filter to     // turn off filter (empty expression)

Some might consider this approach inferior to "scope blocks" or closures of some kind because blocking would guarantee that the filter does not accidently "stay on" if we forgot to close it, but the XBase approach

allows more interaction

. You could use the same commands in interactive mode that you could in the programming code. Thus, with some copy-and-paste one could create a module out of their interactive session with only minor tweaks. One could also use a coded module to set up the environment, but then switch to interactive mode from that point on. That would be harder to do with scoped blocks because one must know the end-point ahead of time.

Another advantage of this is that you can do preliminary set up, such as set a table filter to be viewing a subset of a table, and then "dump" the control back into interactive mode (dot prompt) with such settings left "on". This allows

mixing of scripts and interactive behavior

. Formal closures may not like that because closing would need to be defined or verified up front. Sometimes you want the machine to start the sandwich, but manually finish it (or mix and match sub-scripts manually). It facilitates semi-automation.

Early dialects required a fair amount of work managing indexes, but later dialects moved toward more automatic approaches where indexes were automatically opened and updated when a given table was opened or updated. Unfortunately, the newer approaches were not standardized. Generally the dBASE III+ dialect was probably the final version influencing a universal dialect. They diverged after that (although there was still some cross-borrowing). Thus, if one wanted to target multiple dialects, they usually stuck to III+ conventions.

Although many aspects of the language and tools are clunky, it has some interesting features that are hard to find in SQL-based tools. Many still consider it an excellent ad-hoc data transformation tool because it is usually easier to inspect intermediate steps and do incremental experiments with than with SQL tools. SQL has a more "batch" philosophy behind it.

Format Template Strings

One thing about

ExBase

that greatly simplifies writing the validation in

CrudScreen

applications is its format template strings. You didn't have to write a whole lot of validation code because the format string would guarantee the user entered only what the format would allow. For example, a phone number format template would look something like:

   "(999)999-9999 \xAAAAA"

Here "9" is for digits and "A" is any alpha-numeric char. The "\x" escapes the "x" for phone extension since "X" was also a formatting character. (The specifics may not be entirely accurate here, I don't remember the details.)

The templates appear to be roughly borrowed from Cobol's character templates. The Xbase cursor would not produce a character that was not allowed in a given position. Thus, if you typed a letter where a digit was expected, the cursor simply would not move (and beep or have custom handlers in some dialects). The "marker" characters, such as the parentheses in the phone number helped the user know where to type. The cursor automatically skipped over the marker characters.

It is one of those intuitive, useful, and life-simplifying features that you really miss when you have to do it the hard way. I hope

AjAx

adopts them. Swing has added "J

Formatted

'Text

Field" that allegedly does something similar. Superficially it looks kind of limited.

Don't you mean CSS/HTML instead of AjAx? AjAx is an approach to developing Web applications that reduces page loads by (re)loading specific page elements rather than the whole page. What you want would have to be a feature of CSS/HTML, though it could be created in Javascript.

Pros

  • DomainSpecificLanguage - Language designed with nimble database and business processes in mind
    • What the hell is a "business process"? Sounds like pseudo science and a BuzzPhrase. Nimble database? What the hell is that? Premature optimization?
      • Business Process definition: http://en.wikipedia.org/wiki/Business_process. They were initially described in 1776 by AdamSmith, so I do no think it is a BuzzPhrase.
        • "A business process or business method is a collection of interrelated tasks, which solve a particular issue." A particular issue? Laughing my arse off. Gee, that is almost as clear as mud in my back yard. Sounds almost like saying "we do stuff, to fix stuff, with stuff".
        • See CategoryBusinessDomain for some examples.

      • As a common example of "nimble", I think we can agree that MicrosoftAccess is more "nimble" than Oracle. Oracle is optimized for scalability and reliability, not nimbleness.
        • Business processes are not reliable are they? That's good.. maybe FabianPascal will have some comments about TheWest again. -AnonymousDonor
        • Do you mean actual business rules? They can often change in unexpected ways, yes. Related: AreBusinessAppsBoring.
        • By "reliable" I meant more about database up-time and reduced data corruption, etc. ExBase tools are not well-suited for say a medical prescription tracking system, but rather a marketing database for a small-to-medium company on a tight budget and tight deadline.
      • You're comparing apples to oranges. The apparent nimbleness of MicrosoftAccess is because the user's sole view of MicrosoftAccess is an administrative/development front-end for the (largely) SQL-based Jet database engine. What you see of Access is comparable to TOAD, phpMyAdmin, MySQLAdmin, pgAdminIII, and similar DBA tools. Oracle (at least Oracle Database) is comparable (in terms of nimbleness) to the naked Jet database engine, not MicrosoftAccess.
      • Oracle also has development tools. Perhaps Oracle schema management could be done in a nimble way, but it is usually optimized for performance and availability and the practices in place for it reflect that; while Access is used when they want it soon and cheap. And I'm not ready to compare MS-Access to Toad. True, Toad has a lot of CRUD features, but they are difficult and awkward to use relative to MS-Access (which has it's own warts {pun}, but that's a different topic.)
      • The definition of nimble is quick, fast, agile. Is MS Access quicker than Oracle? Or does nimble refer to quick and dirty solutions that are faster to prepare and get ready than other solutions?
      • As far as machine performance, generally not. Oracle is usually quicker. "Nimble" is in terms of developer effort, at least if up-time and data integrity are not the overriding factors.

  • Cursor-Query-Integation - The cursor-friendly syntax allowed one to get a feel for the data incrimentally, to x-ray intermediate results, and to more easily integrate imperative algorithms with the query process. These are something that I find SQL does not do so smoothly (although it could be made closer to it by allowing user-defined or temp views and user-defined functions). SQL-based tools tend to have a heavy DiscontinuitySpike when mixing or changing between the query language and imperative (IF, looping, temporary variables, user-defined functions, etc). Sometimes imperative is just the TheRightToolForTheJob for some parts of processing. Other tools pretend this is not the case, making it difficult to mix the two concepts. Cursoring is also sometimes more efficient for certain algorithms compared to declarative queries under certain RDBMS. See IteratorVersusQuery for an example. Below is an illustration of updating a table in an imperative loop. This is something that other tools do not do conveniently. (Although an SQL UPDATE statement can do the same thing for simple cases, for more intricate logic it gets either ugly or inefficient, such as when different conditionals result in different behavior and/or when it helps to have a lot of temporary variables to reduce long or repetitious expressions).
         && Loop and Update Example
         USE tableX
         SCAN FOR &myFilterCriteria
            IF x > y AND replacable
               REPLACE x WITH y
            ENDIF
            IF a + b < c
               REPLACE message WITH "Low"
            ENDIF
         ENDSCAN

  • Slim wall between interaction and execution - One could fairly easily hop in and out of interaction versus script execution mode. This is great for experimental projects, debugging, and mixing automation with manual fiddling.

  • Typer friendly - (That's "typeR", not "type") It was designed to avoid having to type too much. This is especially useful for ad-hoc processing. One did not have type lots of qualifiers, quotes, braces, paths, etc. Plus, only the first 4 letters of the commands were necessary (the rest optional). I find that although mouse-centric tools are easier to learn, mousing is generally slower once learned than a well-designed key-board-based tool. Thus, the mouse-centric tools, like MS-Access make it easier to get up and going when first encountered, but one is not as productive under it. (Later versions added mouse-friendly features, but I saw little reason to use them often.)

  • Reuse of commands for automation - ExBase commands can easily be turned into scripts via copy-and-paste (earlier versions did not buffer commands though). It is difficult to script the mouse in say MS-Access, requiring one to switch hats and reinvent the wheel when a repeated process needs scripting. Your prior effort and knowledge for the manual way of doing it gets lost when one goes from mousing to scripting in MS-Access.

  • Integration of app language and query language - SQL-based tools often have a linguistical wall between the SQL and the scripting language that slows one down. ExBase's app language and query language used the same syntax conventions and commands for the most part. RDBMS-vendor-specific app and cursor languages such as Oracles PL/SQL sort of give one a similar feel, but they tend to be more formal and verbose than ExBase. PL/SQL was influenced by Ada, a compiled language, if I'm not mistaken; but ExBase is more scriptish and dynamic in flavor.

  • Easy Reading - Because the sequences were represented as scripts, it's easy to print them out and see what is going on all on a sheet or two of paper. With MicrosoftAccess, you have to open and close gazillion different objects to follow the sequence.

  • Easy Schema Manipulation - It was easy to turn a table into a schema and a schema into a table. Thus, schemas and tables could be easily generated on-the-fly.

  • Programmer-Controlled Joining - Sometimes one needed more control over the join/look-up process for custom fiddling or decisions. ExBase made it fairly easy to "stick one's fingers into" the look-up process to add special actions.

  • Each table was its own file. This made mix-and-match easier, and limited corruption problems to a single table.

Cons

  • Index managing was either too manual, or differed per vendor. (This was less of problem with ad-hoc queries because one does not keep the indexes created anyhow.)

  • Poor or lacking scope-management features: Its "auto-context" and mode-based scoping nature that simplified things for smaller processes made managing larger code bases more difficult. Thus, it was not a very good tool for writing complex packaged software developed by larger teams. (This is also partly because people tried to copy C's style rather than take advantage of table-oriented frameworks. One had to learn to "think in tables" to make better use of its idioms.)

  • Either did not offer variable-length columns, or did not implement them well.

  • Joins were usually more difficult in "cursor mode", but later versions let one use SQL for that.

  • Variable names were limited in length (extra characters were ignored). However, this is not a fault of the concept, but merely a reflection of the resource-limited hardware it was built for at the time. (It was an amazing tool for something that fit in less than 100k for the earlier versions.)

  • Lacked many referential integrity features. (Later additions varied too much between vendors.)

  • It's multi-file approach made copying and transferring entire projects a bit more difficult than the single-file approach MS-Access and friends use.

  • Global modes made developing modular or reusable code maddeningly difficult and the result painfully inefficient. Global modes had to be carefully preserved, set and restored at each entry and exit point or at every point where modes had an effect. This staggering idiocy single-handedly guaranteed that professional developers would never take xBase products seriously. Defence of this "feature", or at least failure to recognise it as utterly dire, was sometimes used in interview processes to eliminate questionable candidates.

    • If used in proper context for the tool, this didn't matter nearly as much as you claim. Most re-use I do now is to handle low-level issues that ExBase took care of already in the language/tool. Programs were often 1/5 the code of the equivalent of say C-sharp. At least it made reinventing the wheel really easy. It made an excellent prototyping tool at least. Once the requirements and features settle, then one can use one of the more anal and verbose languages to make a "team-friendly" version. Your one-programming-style-fits-all-projects mentality is concerning. -t

    • There is no such thing as "proper context" for a global mode in a language, unless it's something intended to apply universally to all running and runnable applications such as where error messages get logged. Then it's normally a system-wide configuration setting that isn't accessible to the language itself. The problem with xBase global modes is that they could be set under program control and they affected behaviour of individual commands. That meant one module could break another by changing a mode, unless every module was sufficiently disciplined to explicitly set modes, which had a performance impact on an already slow language.

    • Out-of-the box xBase programs were 1/5 the code of out-of-the-box C# programs, but nobody uses out-of-the-box C# for production development. You build or buy libraries to extend it, which means your C# programs become 1/5 the size of xBase programs whilst supporting code re-use. Of course, that's not only the case with C#. Back in xBase's heydey, we used C/C++ to target our xBase-using competitors and eliminate them by being able to deliver faster, slicker, and more reliable applications sooner and at lower cost. xBase was somewhat handy for quick-n-dirty prototyping, but we rarely used it because we had essentially "production-lined" the development of typical record-keeping, reporting and accounting applications.

    • That differs from my observations. Yes, if you want fine control over the UI and want to standardize carefully-crafted widgets and behaviors into libraries, then xBase is indeed not the right thing. UI fine-control was not its strong point. C/C++ contractors took too long and were too expensive; and they crashed a lot due to pointer errors. As long as you level with the customer about the trade-offs, xBase had the niche for getting it done. C/C++ teams had slick UI widgets (for the DOS era), but were slow and follow-on contractors took too long to figure out the prior contractor's libraries. If you use xBase's built in features, then everybody knows them already. I'm not saying the global context thing is a good design, only that you are exaggerating the downsides. If a shop adopted standard practices, their downsides can be contained.

    • I suppose if you don't mind the general poor quality of xBase applications and expect random contractors to work on them, then xBase is a reasonable choice. It's the same niche that MicrosoftAccess fills now.

    • Some of the highest praise I've ever received in my career was over xBase apps. And generally I do agree with your statement about MicrosoftAccess, but it's largely because xBase failed to adapt well to GUI's, not because it was a bad general idea. I agree much of the "scope leak" should be cleaned up in the language, but I remember the good parts at least as much as the bad. It was on to something. MS-Access is full of warts. -t

    • "Some of the highest praise I've ever received in my career was over xBase apps." You're proud of that? Some of the highest praise I've ever received in my career was over stuff I did yesterday. I'm hoping to get even higher praise tomorrow over the stuff I'm doing today. If you're defining your career achievements by what you did in a dead language twenty-five years ago... Oh dear. If you think xBase was onto something, then learn about parsing and interpretation and build a better xBase. It's not difficult, and that would be something to be proud of.

    • I'll readily admit I was more productive under xBase, at least in terms of the tools/technology of the day. My own efficiency put me out of my own job. As far as improving on it, topics such as MaspBrainstorming, MaspImplementationDiscussion, SmeQl, and DynamicRelational are influenced by xBase. I'm just not very adept at SystemsSoftware (perhaps because I'm not skilled at HOF's :-) One of these days at least I may get around to a prototype.

    • How did your own efficiency put you out of a job?

      • My job was custom apps for accounting, marketing, and project status tracking, much if it migrating off of mini's and mainframes. I wrapped those up pretty quick and they needed relatively little maintenance. Thus, they started giving me more desktop PC equipment and OS support tasks, which is not where I wanted to focus my career, and so left.

      • Why didn't they give you more programming tasks? I've never seen a business with a programmer that couldn't keep him or her continuously busy coding new applications or adding functionality to old ones.

      • The bureaucracy and office politics reduced cross-department project sharing there. Plus, they needed a PC fixer there. My apps were reliable, PC's and Novel weren't.

    • Also, ControlTables are difficult to do in C/C++ because they lack Eval, for one.

    • Almost trivial, though, if you embed a scripting sublanguage like Lua, Javascript, TinyC or whatever -- and use that to give your end-users scripting control -- instead of trying to put C++ in your ControlTables. Alternatively, write the whole thing in PHP, Python, Perl or Ruby. We used C/C++ in the late 80s because there weren't many choices. There are many more choices now.

    • Indeed. Almost nobody does run-of-the-mill CBA apps in C/C++ these days.

  • Filesystem-based multi-user support. This limited scalability, compromised reliability (network failures could leave "transactions" in a partially-completed state), and in most ExBase implementations required the developer to explicitly code (frequently complex) locking and transaction mechanisms.

What ExBase Taught Me:

  • Databasing doesn't have to be stiff and verbose like BigIron databasing. There can be "scripty" databasing also. (I wish there were more language people working on such a niche.)

  • DataAndCodeAreTheSameThing

  • The value of using tables to "drive" an app instead of lines of hard-wired programming code, in an almost FunctionalProgramming way (See TableOrientedProgramming and ControlTable). ExBase allowed me to easily experiment with such and hone my techniques.

  • The value of easily mixing imperative and declarative so that the best technique for the job can be used without a thick translating layer between them. This is something that SQL struggles with because the app language is a different universe than the database language. Database languages like Pl/Sql kind of have such integration, but are more verbose and "stiff" than xBase. (I agree that such "protection" code in Pl/Sql can help in bigger team environments, but not small, ad-hoc, or throw-away projects.)

  • The instant gratification value of compact but powerful commands that can be used in both command-line mode and in script files. Contrast with the GUI world of MS Access where your command knowledge and GUI knowledge are two different worlds you have to learn and remember. The blurring of command mode and coding mode also makes debugging more natural because you already know the command line mode and environment.

  • The benefits and pitfalls of relying on context and global "modes" to reduce verbosity. It's a great code-simplifying technique, but can make a huge mess if not used with care. It's probably better used for ad-hoc one-off programming. But, when you are actually doing ad-hoc programming, it's a very powerful time-saver. A HackerLanguage has it's place.
    • The "mode" approach also simplifies the command-line-to-script cross-mixing capability (prior item above) because at the command line, block- or closure-based scope or context management doesn't work so well. The end-points have to be "pre-known" to use blocks/closures. At the command line, a mode or context stays in place until explicitly closed or set to "off". In theory it could be "left open/on" forever.

  • The value of easily generating tables from schemas and schemas from tables on the fly. It often made debugging and process partitioning easier to create intermediate or temp "work" tables. SQL often forces one to try to chew everything in one big bite instead of taking advantage of DivideAndConquer. (The relatively new WITH clause in the SQL standard helps, but it's still kind of verbose.)

  • It almost mastered and demostrated TheRightWayToDoWordyBlocks. You don't have to use the Shift key very often typing xBase.

  • EconomyOfExpression: Less code and less typing that does more. See near the bottom of UniversalStatement regarding shortcuts for Boolean named parameters.

  • TheRightToolForTheJob - The best design and syntax for smaller or ad-hoc projects is not necessarily the same for bigger "team" or sensitive operation projects.

--top

If anybody knows of a decent

open-source interpretive ExBase

, I'd be glad to get my hands on it. I really really miss

ExBase

for ad-hoc processing and "self" scripting.

Even Harbour (see http://harbour.github.io/), an ExBase compiler that came as close as anything to sustaining ExBase in OpenSource form, seems to have gone dead. Maybe you could build it?

Links

Appreciation site:

http://www.geocities.com/tablizer/xbasefan.htm

Interview with creator:

http://reddevnews.com/qandas/article.aspx?editorialsid=113

Wayne C. Ratliff on dBASE's origins:

http://www.accessmylibrary.com/coms2/summary_0286-9229460_ITM

(An interesting read for anybody wanting to start a company.)

See Also:

CollectionOrientedProgramming

,

PowerfulAdHocDataProcessingTools

,

ModernizingExBase

,

ExBaseRant

,

XbaseLibrary CategoryOldSoftware

,

CategorySoftwareTool

,

CategoryDatabase

,

CategoryScripting

,

CategoryClosure

(reasons not to)

Continue reading on c2.com